[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:32:56 dist-test-slave-phbh krb5kdc[7014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:32:56 dist-test-slave-phbh krb5kdc[7014](info): set up 2 sockets
Sep 26 19:32:56 dist-test-slave-phbh krb5kdc[7014](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 19:32:57.547827 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.002s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:32:57 dist-test-slave-phbh krb5kdc[7014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:32:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:32:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:32:57.723061 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:35629
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:35629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:32:58.043406 7027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:32:58.043720 7027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:32:58.043874 7027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:32:58.051321 7027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:32:58.051445 7027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:32:58.051529 7027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:32:58.051622 7027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:32:58.051713 7027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:32:58.061681 7027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:35629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:35629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:32:58.063074 7027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:32:58.064409 7027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:32:58.076821 7033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:58.076922 7032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:58.076821 7035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:32:58.077183 7027 server_base.cc:1034] running on GCE node
I20260926 19:32:58.078207 7027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:32:58.079567 7027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:32:58.080762 7027 hybrid_clock.cc:648] HybridClock initialized: now 1790451178080682 us; error 67 us; skew 500 ppm
I20260926 19:32:58.081156 7027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:32:58 dist-test-slave-phbh krb5kdc[7014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451178, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:32:58.084160 7027 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:32:58.085717 7027 webserver.cc:460] Webserver started at http://127.6.215.254:40249/ using document root <none> and password file <none>
I20260926 19:32:58.086400 7027 fs_manager.cc:359] Metadata directory not provided
I20260926 19:32:58.086526 7027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:32:58.086826 7027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:32:58.090221 7027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "d6563dd5874c4dfaaf2a3d3db47892d6"
format_stamp: "Formatted at 2026-09-26 19:32:58 on dist-test-slave-phbh"
server_key: "6dc93d6b14b4a381f7608d0fca9fc640"
server_key_iv: "0b855e07cd84d293aba2456ba5dcfcf8"
server_key_version: "encryptionkey@0"
I20260926 19:32:58.090939 7027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "d6563dd5874c4dfaaf2a3d3db47892d6"
format_stamp: "Formatted at 2026-09-26 19:32:58 on dist-test-slave-phbh"
server_key: "6dc93d6b14b4a381f7608d0fca9fc640"
server_key_iv: "0b855e07cd84d293aba2456ba5dcfcf8"
server_key_version: "encryptionkey@0"
I20260926 19:32:58.096133 7027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:32:58.099509 7042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:58.100813 7027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:32:58.101022 7027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "d6563dd5874c4dfaaf2a3d3db47892d6"
format_stamp: "Formatted at 2026-09-26 19:32:58 on dist-test-slave-phbh"
server_key: "6dc93d6b14b4a381f7608d0fca9fc640"
server_key_iv: "0b855e07cd84d293aba2456ba5dcfcf8"
server_key_version: "encryptionkey@0"
I20260926 19:32:58.101227 7027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:32:58.129755 7027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:32:58.134117 7027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:32:58.134435 7027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:32:58.153785 7027 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:35629
I20260926 19:32:58.153807 7093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:35629 every 8 connection(s)
I20260926 19:32:58.155166 7027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:32:58.159325 7094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:32:58.159822 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7027
I20260926 19:32:58.160116 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:32:58.161998 7007 external_mini_cluster.cc:1428] Setting key 47e317413e9e89abdd4aa725e0b5ec6a
I20260926 19:32:58.169761 7094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6: Bootstrap starting.
I20260926 19:32:58.172711 7094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6: Neither blocks nor log segments found. Creating new log.
I20260926 19:32:58.173733 7094 log.cc:824] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6: Log is configured to *not* fsync() on all Append() calls
I20260926 19:32:58.176719 7094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6: No bootstrap required, opened a new log
I20260926 19:32:58.186436 7094 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 35629 } }
I20260926 19:32:58.186896 7094 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:32:58.187021 7094 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6563dd5874c4dfaaf2a3d3db47892d6, State: Initialized, Role: FOLLOWER
I20260926 19:32:58.187714 7094 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 35629 } }
I20260926 19:32:58.188027 7094 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:32:58.188191 7094 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:32:58.188378 7094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:32:58.191669 7094 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 35629 } }
Sep 26 19:32:58 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:32:58.192174 7094 leader_election.cc:302] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6563dd5874c4dfaaf2a3d3db47892d6; no voters:
I20260926 19:32:58.192891 7094 leader_election.cc:288] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:32:58.193059 7099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:32:58.195019 7094 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:32:58.196703 7099 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [term 1 LEADER]: Becoming Leader. State: Replica: d6563dd5874c4dfaaf2a3d3db47892d6, State: Running, Role: LEADER
I20260926 19:32:58.197410 7099 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 35629 } }
I20260926 19:32:58.200456 7100 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 35629 } } }
I20260926 19:32:58.200924 7100 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:32:58.209122 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:58.165721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.6.215.254:35629)
0926 19:32:58.166619 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:58.166637 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:58.167947 (+ 1310us) server_negotiation.cc:407] Connection header received
0926 19:32:58.176649 (+ 8702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:58.176670 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:58.177134 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:58.177475 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:58.181367 (+ 3892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:58.182890 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:58.185330 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:58.186067 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:58.186211 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:58.193726 (+ 7515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:58.193744 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:58.193773 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:58.193854 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:58.199242 (+ 5388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:58.200602 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:58.200611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:58.200621 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:58.200888 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:58.201659 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:58.201668 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:58.201675 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:58.202674 (+ 999us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:58.206094 (+ 3420us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:58.206585 (+ 491us) server_negotiation.cc:299] Negotiation successful
0926 19:32:58.207174 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":389,"threads_started":1}
I20260926 19:32:58.209549 7107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:32:58.209915 7101 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6563dd5874c4dfaaf2a3d3db47892d6. Latest consensus state: current_term: 1 leader_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6563dd5874c4dfaaf2a3d3db47892d6" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 35629 } } }
I20260926 19:32:58.210283 7101 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6 [sys.catalog]: This master's current role is: LEADER
I20260926 19:32:58.217054 7107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:32:58.226464 7107 catalog_manager.cc:1347] Generated new cluster ID: e0bfdbfc60704046a6764ece58f3924b
I20260926 19:32:58.226584 7107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:32:58.244095 7107 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:32:58.246214 7107 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:32:58.262789 7107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6563dd5874c4dfaaf2a3d3db47892d6: Generated new TSK 0
I20260926 19:32:58.264149 7107 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:32:58.324671 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:35629
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:32:58.652396 7121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:32:58.652667 7121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:32:58.652880 7121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:32:58.660226 7121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:32:58.660334 7121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:32:58.660544 7121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:32:58.670202 7121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:35629
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:32:58.671458 7121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:32:58.672900 7121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:32:58.685829 7126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:58.685918 7127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:58.685940 7129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:32:58.686906 7121 server_base.cc:1034] running on GCE node
I20260926 19:32:58.687525 7121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:32:58.688488 7121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:32:58.689716 7121 hybrid_clock.cc:648] HybridClock initialized: now 1790451178689650 us; error 86 us; skew 500 ppm
I20260926 19:32:58.690044 7121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:32:58 dist-test-slave-phbh krb5kdc[7014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451178, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:32:58.693285 7121 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:32:58.694734 7121 webserver.cc:460] Webserver started at http://127.6.215.193:45859/ using document root <none> and password file <none>
I20260926 19:32:58.695561 7121 fs_manager.cc:359] Metadata directory not provided
I20260926 19:32:58.695701 7121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:32:58.696036 7121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:32:58.698992 7121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "8c490338de8243509e5d3ebfa1603e2e"
format_stamp: "Formatted at 2026-09-26 19:32:58 on dist-test-slave-phbh"
server_key: "a5c2d506cd2b1fbc496c61c85938a6a2"
server_key_iv: "79d4880bf13ce5317f1dbf85ff22a342"
server_key_version: "encryptionkey@0"
I20260926 19:32:58.699760 7121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "8c490338de8243509e5d3ebfa1603e2e"
format_stamp: "Formatted at 2026-09-26 19:32:58 on dist-test-slave-phbh"
server_key: "a5c2d506cd2b1fbc496c61c85938a6a2"
server_key_iv: "79d4880bf13ce5317f1dbf85ff22a342"
server_key_version: "encryptionkey@0"
I20260926 19:32:58.704985 7121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 19:32:58.708652 7136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:58.710027 7121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:32:58.710196 7121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "8c490338de8243509e5d3ebfa1603e2e"
format_stamp: "Formatted at 2026-09-26 19:32:58 on dist-test-slave-phbh"
server_key: "a5c2d506cd2b1fbc496c61c85938a6a2"
server_key_iv: "79d4880bf13ce5317f1dbf85ff22a342"
server_key_version: "encryptionkey@0"
I20260926 19:32:58.710413 7121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:32:58.723531 7121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:32:58.727830 7121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:32:58.728104 7121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:32:58.729259 7121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:32:58.730522 7121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:32:58.730624 7121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:58.730765 7121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:32:58.730849 7121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:58.764575 7121 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:33515
I20260926 19:32:58.764623 7248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:33515 every 8 connection(s)
I20260926 19:32:58.765971 7121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:32:58.773733 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7121
I20260926 19:32:58.773967 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:32:58.774751 7007 external_mini_cluster.cc:1428] Setting key 8fe8ff2ce701359663464be273128c88
Sep 26 19:32:58 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451178, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:32:58.788601 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:58.768415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:48339 (local address 127.6.215.254:35629)
0926 19:32:58.768861 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:58.768868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:58.769801 (+ 933us) server_negotiation.cc:407] Connection header received
0926 19:32:58.771466 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:58.771470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:58.771533 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:58.771628 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:58.773138 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:58.773899 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:58.776113 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:58.776724 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:58.776890 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:58.781260 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:58.781266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:58.781269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:58.781366 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:58.785137 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:58.785882 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:58.785891 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:58.785898 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:58.786066 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:58.786485 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:58.786494 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:58.786501 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:58.786908 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:58.787077 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:58.787734 (+ 657us) server_negotiation.cc:299] Negotiation successful
0926 19:32:58.788052 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":207,"threads_started":1}
I20260926 19:32:58.789124 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:58.768981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:35629 (local address 127.6.215.193:48339)
0926 19:32:58.769640 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0926 19:32:58.769683 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:32:58.771228 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:32:58.771834 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:32:58.771841 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:32:58.772340 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:32:58.772985 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:32:58.773003 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:32:58.774071 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:32:58.774076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:32:58.775983 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:32:58.775998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:32:58.776986 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:32:58.776994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:32:58.777274 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:58.777967 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:32:58.777999 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:32:58.780994 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:32:58.785311 (+ 4317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:32:58.785317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:32:58.785329 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:32:58.785735 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:32:58.786182 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:32:58.786186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:32:58.786188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:32:58.786355 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:32:58.787032 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:32:58.787037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:32:58.787338 (+ 301us) client_negotiation.cc:769] Sending connection context
0926 19:32:58.787548 (+ 210us) client_negotiation.cc:240] Negotiation successful
0926 19:32:58.787908 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":255,"threads_started":1}
I20260926 19:32:58.790727 7249 heartbeater.cc:347] Connected to a master server at 127.6.215.254:35629
I20260926 19:32:58.791090 7249 heartbeater.cc:464] Registering TS with master...
I20260926 19:32:58.792064 7249 heartbeater.cc:511] Master 127.6.215.254:35629 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:32:58.794528 7059 ts_manager.cc:194] Registered new tserver with Master: 8c490338de8243509e5d3ebfa1603e2e (127.6.215.193:33515)
I20260926 19:32:58.796187 7059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:48339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:32:58.818296 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:35629
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:32:59.132972 7255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:32:59.133291 7255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:32:59.133468 7255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:32:59.140589 7255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:32:59.140781 7255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:32:59.140996 7255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:32:59.150558 7255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:35629
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:32:59.151822 7255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:32:59.153304 7255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:32:59.167971 7261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:59.167919 7260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:59.168396 7263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:32:59.168860 7255 server_base.cc:1034] running on GCE node
I20260926 19:32:59.169421 7255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:32:59.170333 7255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:32:59.171561 7255 hybrid_clock.cc:648] HybridClock initialized: now 1790451179171502 us; error 49 us; skew 500 ppm
I20260926 19:32:59.171862 7255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:32:59.175053 7255 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:32:59.176502 7255 webserver.cc:460] Webserver started at http://127.6.215.194:45791/ using document root <none> and password file <none>
I20260926 19:32:59.177220 7255 fs_manager.cc:359] Metadata directory not provided
I20260926 19:32:59.177366 7255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:32:59.177668 7255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:32:59.180466 7255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "b4a86048053940a1a8d830604fad09e1"
format_stamp: "Formatted at 2026-09-26 19:32:59 on dist-test-slave-phbh"
server_key: "92222deb09eb6177e55785d8d2cb0a87"
server_key_iv: "e37630051ad1b93aa01c38b04b535557"
server_key_version: "encryptionkey@0"
I20260926 19:32:59.181222 7255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "b4a86048053940a1a8d830604fad09e1"
format_stamp: "Formatted at 2026-09-26 19:32:59 on dist-test-slave-phbh"
server_key: "92222deb09eb6177e55785d8d2cb0a87"
server_key_iv: "e37630051ad1b93aa01c38b04b535557"
server_key_version: "encryptionkey@0"
I20260926 19:32:59.186254 7255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:32:59.189750 7270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:59.191234 7255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:32:59.191478 7255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "b4a86048053940a1a8d830604fad09e1"
format_stamp: "Formatted at 2026-09-26 19:32:59 on dist-test-slave-phbh"
server_key: "92222deb09eb6177e55785d8d2cb0a87"
server_key_iv: "e37630051ad1b93aa01c38b04b535557"
server_key_version: "encryptionkey@0"
I20260926 19:32:59.191697 7255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:32:59.212185 7255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:32:59.217053 7255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:32:59.217344 7255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:32:59.218345 7255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:32:59.219694 7255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:32:59.219765 7255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:59.219935 7255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:32:59.219987 7255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:59.250885 7255 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:33293
I20260926 19:32:59.250923 7382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:33293 every 8 connection(s)
I20260926 19:32:59.252211 7255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:32:59.255381 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7255
I20260926 19:32:59.255570 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:32:59.256006 7007 external_mini_cluster.cc:1428] Setting key b80807c123c14b5dcf7daff2f8e120ad
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:32:59.273998 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.254867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:34549 (local address 127.6.215.254:35629)
0926 19:32:59.255043 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:59.255047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:59.256268 (+ 1221us) server_negotiation.cc:407] Connection header received
0926 19:32:59.258542 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:59.258548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:59.258633 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:59.258736 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:59.260125 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.261337 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.263630 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.264029 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.264129 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.268293 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:59.268300 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:59.268304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:59.268338 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:59.271049 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.271775 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.271780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.271784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.271913 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.272389 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.272395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.272398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.272659 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:59.272856 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:59.273585 (+ 729us) server_negotiation.cc:299] Negotiation successful
0926 19:32:59.273746 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
I20260926 19:32:59.275012 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.255255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:35629 (local address 127.6.215.194:34549)
0926 19:32:59.256083 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0926 19:32:59.256127 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 19:32:59.258267 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:32:59.258942 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:32:59.258949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:32:59.259440 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:32:59.259958 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:32:59.259973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.261478 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.261485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:32:59.263495 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:32:59.263512 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.264248 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.264257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:32:59.264568 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.265281 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:32:59.265319 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:32:59.268049 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:32:59.271200 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:32:59.271208 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:32:59.271224 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:32:59.271645 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:32:59.272040 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:32:59.272044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:32:59.272048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:32:59.272272 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:32:59.272883 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:32:59.272890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:32:59.273220 (+ 330us) client_negotiation.cc:769] Sending connection context
0926 19:32:59.273492 (+ 272us) client_negotiation.cc:240] Negotiation successful
0926 19:32:59.273867 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":351,"threads_started":1}
I20260926 19:32:59.276515 7383 heartbeater.cc:347] Connected to a master server at 127.6.215.254:35629
I20260926 19:32:59.276862 7383 heartbeater.cc:464] Registering TS with master...
I20260926 19:32:59.277733 7383 heartbeater.cc:511] Master 127.6.215.254:35629 requested a full tablet report, sending...
I20260926 19:32:59.279248 7059 ts_manager.cc:194] Registered new tserver with Master: b4a86048053940a1a8d830604fad09e1 (127.6.215.194:33293)
I20260926 19:32:59.280042 7059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:34549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:32:59.296413 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:35629
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:32:59.616968 7388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:32:59.617390 7388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:32:59.617575 7388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:32:59.625309 7388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:32:59.625416 7388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:32:59.625530 7388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:32:59.635610 7388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:35629
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:32:59.636761 7388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:32:59.638635 7388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:32:59.652630 7393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:59.652693 7394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:32:59.652661 7396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:32:59.654018 7388 server_base.cc:1034] running on GCE node
I20260926 19:32:59.654594 7388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:32:59.655485 7388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:32:59.656706 7388 hybrid_clock.cc:648] HybridClock initialized: now 1790451179656676 us; error 50 us; skew 500 ppm
I20260926 19:32:59.657033 7388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:32:59.660049 7388 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:32:59.661547 7388 webserver.cc:460] Webserver started at http://127.6.215.195:40777/ using document root <none> and password file <none>
I20260926 19:32:59.662197 7388 fs_manager.cc:359] Metadata directory not provided
I20260926 19:32:59.662338 7388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:32:59.662645 7388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:32:59.665494 7388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "9890613402054261989d0090077f09d3"
format_stamp: "Formatted at 2026-09-26 19:32:59 on dist-test-slave-phbh"
server_key: "6fc094ff12a70e615e5477f5fdae4f41"
server_key_iv: "f5a4669f46f6223549eaaddd6bc274d0"
server_key_version: "encryptionkey@0"
I20260926 19:32:59.666241 7388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "9890613402054261989d0090077f09d3"
format_stamp: "Formatted at 2026-09-26 19:32:59 on dist-test-slave-phbh"
server_key: "6fc094ff12a70e615e5477f5fdae4f41"
server_key_iv: "f5a4669f46f6223549eaaddd6bc274d0"
server_key_version: "encryptionkey@0"
I20260926 19:32:59.671118 7388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 19:32:59.674391 7403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:59.675643 7388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:32:59.675805 7388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "9890613402054261989d0090077f09d3"
format_stamp: "Formatted at 2026-09-26 19:32:59 on dist-test-slave-phbh"
server_key: "6fc094ff12a70e615e5477f5fdae4f41"
server_key_iv: "f5a4669f46f6223549eaaddd6bc274d0"
server_key_version: "encryptionkey@0"
I20260926 19:32:59.676010 7388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:32:59.689507 7388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:32:59.693375 7388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:32:59.693646 7388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:32:59.694512 7388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:32:59.695993 7388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:32:59.696085 7388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:59.696219 7388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:32:59.696285 7388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:32:59.728282 7388 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:35467
I20260926 19:32:59.728328 7515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:35467 every 8 connection(s)
I20260926 19:32:59.729543 7388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:32:59.732221 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7388
I20260926 19:32:59.732415 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:32:59.732959 7007 external_mini_cluster.cc:1428] Setting key 45eabed5388d244b747e5ddfd784656b
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:32:59.749091 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.732136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54461 (local address 127.6.215.254:35629)
0926 19:32:59.732373 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:59.732381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:59.733724 (+ 1343us) server_negotiation.cc:407] Connection header received
0926 19:32:59.735571 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:59.735576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:59.735658 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:59.735776 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:59.737368 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.738174 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.739695 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.740106 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.740238 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.743591 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:59.743601 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:59.743605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:59.743635 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:59.746127 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.746827 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.746837 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.746840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.746973 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.747405 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.747414 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.747417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.747667 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:59.747813 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:59.748480 (+ 667us) server_negotiation.cc:299] Negotiation successful
0926 19:32:59.748683 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:32:59.749969 7518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.732701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:35629 (local address 127.6.215.195:54461)
0926 19:32:59.733565 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0926 19:32:59.733617 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 19:32:59.735299 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:32:59.736014 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:32:59.736023 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:32:59.736579 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:32:59.737205 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:32:59.737224 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.738338 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.738347 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:32:59.739552 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:32:59.739563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.740317 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.740327 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:32:59.740565 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.741209 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:32:59.741230 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:32:59.743390 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:32:59.746268 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:32:59.746277 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:32:59.746292 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:32:59.746698 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:32:59.747087 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:32:59.747093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:32:59.747096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:32:59.747278 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:32:59.747802 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:32:59.747810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:32:59.748125 (+ 315us) client_negotiation.cc:769] Sending connection context
0926 19:32:59.748379 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 19:32:59.748905 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":354,"threads_started":1}
I20260926 19:32:59.751183 7516 heartbeater.cc:347] Connected to a master server at 127.6.215.254:35629
I20260926 19:32:59.751499 7516 heartbeater.cc:464] Registering TS with master...
I20260926 19:32:59.752163 7516 heartbeater.cc:511] Master 127.6.215.254:35629 requested a full tablet report, sending...
I20260926 19:32:59.753687 7059 ts_manager.cc:194] Registered new tserver with Master: 9890613402054261989d0090077f09d3 (127.6.215.195:35467)
I20260926 19:32:59.754494 7059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:54461
I20260926 19:32:59.760422 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:32:59.799417 7249 heartbeater.cc:503] Master 127.6.215.254:35629 was elected leader, sending a full tablet report...
I20260926 19:32:59.804127 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.788677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.6.215.193:33515)
0926 19:32:59.789335 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:59.789343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:59.789409 (+ 66us) server_negotiation.cc:407] Connection header received
0926 19:32:59.789470 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:59.789476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:59.789683 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:59.789812 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:59.791309 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.792290 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.793618 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.794158 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.794252 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.798394 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:59.798405 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:59.798418 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:59.798452 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:59.801772 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.802417 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.802424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.802426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.802532 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.802836 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.802839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.802841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.803140 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:59.803350 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:59.803575 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 19:32:59.803751 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":347,"threads_started":1}
W20260926 19:32:59.805510 7164 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39006
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:32:59.848126 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.834835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39010 (local address 127.6.215.193:33515)
0926 19:32:59.835078 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:59.835084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:59.835306 (+ 222us) server_negotiation.cc:407] Connection header received
0926 19:32:59.835453 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:59.835457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:59.835520 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:59.835604 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:59.836888 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.837564 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.838778 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.839195 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.839288 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.843005 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:59.843010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:59.843013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:59.843044 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:59.845690 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.846280 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.846284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.846287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.846412 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.846750 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.846752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.846754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.846989 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:59.847153 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:59.847589 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 19:32:59.847758 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 19:32:59.849720 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7121
I20260926 19:32:59.862059 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.850651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.6.215.193:33515)
0926 19:32:59.850879 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:59.850889 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:59.851149 (+ 260us) server_negotiation.cc:407] Connection header received
0926 19:32:59.851262 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:59.851265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:59.851318 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:59.851416 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:59.852458 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.853193 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.854502 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.854897 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.854987 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.857125 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:59.857129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:59.857132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:59.857159 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:59.859663 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.860206 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.860209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.860212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.860326 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.860740 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.860744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.860746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.861090 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:59.861274 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:59.861544 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 19:32:59.861688 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 19:32:59.862761 7244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:32:59.945081 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7121
I20260926 19:32:59.968067 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7255
Sep 26 19:32:59 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:32:59.982972 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:32:59.969038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56252 (local address 127.6.215.194:33293)
0926 19:32:59.969537 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0926 19:32:59.969545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:32:59.969570 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:32:59.969627 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:32:59.969634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:32:59.969834 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:32:59.969975 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:32:59.971089 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.972075 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.973421 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:32:59.973934 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:32:59.974039 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:32:59.977902 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:32:59.977913 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:32:59.977926 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:32:59.977962 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:32:59.980573 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.981290 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.981297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.981299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.981411 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:32:59.981741 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:32:59.981744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:32:59.981746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:32:59.982044 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:32:59.982259 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 19:32:59.982502 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 19:32:59.982677 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":291,"threads_started":1}
I20260926 19:32:59.984270 7378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:00.065079 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7255
I20260926 19:33:00.081701 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7388
Sep 26 19:33:00 dist-test-slave-phbh krb5kdc[7014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:00.098132 7532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:00.082714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.6.215.195:35467)
0926 19:33:00.083347 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:00.083354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:00.083383 (+ 29us) server_negotiation.cc:407] Connection header received
0926 19:33:00.083439 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:00.083446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:00.083654 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:00.083782 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:00.085009 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:00.086022 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:00.087235 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:00.087766 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:00.087861 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:00.092028 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:00.092041 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:00.092057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:00.092091 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:00.095212 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:00.095977 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:00.095985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:00.095988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:00.096158 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:00.096524 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:00.096527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:00.096529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:00.096963 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:00.097207 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:00.097530 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 19:33:00.097727 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":349,"threads_started":1}
I20260926 19:33:00.099538 7511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:00.187621 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7388
I20260926 19:33:00.203910 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7027
I20260926 19:33:00.205250 7089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:00.306583 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7027
2026-09-26T19:33:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3908 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:00 dist-test-slave-phbh krb5kdc[7537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:00 dist-test-slave-phbh krb5kdc[7537](info): set up 2 sockets
Sep 26 19:33:00 dist-test-slave-phbh krb5kdc[7537](info): commencing operation
krb5kdc: starting...
W20260926 19:33:01.377885 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.001s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:01 dist-test-slave-phbh krb5kdc[7537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:01.568297 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42657
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:01.885766 7550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:01.886071 7550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:01.886159 7550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:01.893740 7550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:01.893869 7550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:01.893918 7550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:01.893970 7550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:01.894016 7550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:01.904171 7550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:01.905511 7550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:01.906730 7550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:01.919010 7558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:01.919045 7556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:01.919023 7555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:01.919661 7550 server_base.cc:1034] running on GCE node
I20260926 19:33:01.921861 7550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:01.923501 7550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:01.924892 7550 hybrid_clock.cc:648] HybridClock initialized: now 1790451181924729 us; error 60 us; skew 500 ppm
I20260926 19:33:01.925258 7550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:01 dist-test-slave-phbh krb5kdc[7537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451181, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:01.931869 7550 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:01.933617 7550 webserver.cc:460] Webserver started at http://127.6.215.254:38995/ using document root <none> and password file <none>
I20260926 19:33:01.934443 7550 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:01.934597 7550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:01.934918 7550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:01.938052 7550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "d8f503740a52482bb585b7846b333267"
format_stamp: "Formatted at 2026-09-26 19:33:01 on dist-test-slave-phbh"
server_key: "0427b420c62ec217b487746b7875fac3"
server_key_iv: "d705702e3ec1f612e3dafabc9919d136"
server_key_version: "encryptionkey@0"
I20260926 19:33:01.938865 7550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "d8f503740a52482bb585b7846b333267"
format_stamp: "Formatted at 2026-09-26 19:33:01 on dist-test-slave-phbh"
server_key: "0427b420c62ec217b487746b7875fac3"
server_key_iv: "d705702e3ec1f612e3dafabc9919d136"
server_key_version: "encryptionkey@0"
I20260926 19:33:01.944401 7550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 19:33:01.948251 7565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:01.949862 7550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:33:01.950066 7550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "d8f503740a52482bb585b7846b333267"
format_stamp: "Formatted at 2026-09-26 19:33:01 on dist-test-slave-phbh"
server_key: "0427b420c62ec217b487746b7875fac3"
server_key_iv: "d705702e3ec1f612e3dafabc9919d136"
server_key_version: "encryptionkey@0"
I20260926 19:33:01.950294 7550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:01.988587 7550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:01.993227 7550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:01.993534 7550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:02.013056 7550 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42657
I20260926 19:33:02.013077 7616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42657 every 8 connection(s)
I20260926 19:33:02.014526 7550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:02.016116 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7550
I20260926 19:33:02.016358 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:02.017199 7007 external_mini_cluster.cc:1428] Setting key 2e0d9e0aec04e83d9ead5e41525fd0e9
I20260926 19:33:02.019558 7617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 19:33:02 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:02.033124 7617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267: Bootstrap starting.
I20260926 19:33:02.036365 7617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:02.037459 7617 log.cc:824] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:02.040349 7617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267: No bootstrap required, opened a new log
I20260926 19:33:02.040659 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:02.019446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58764 (local address 127.6.215.254:42657)
0926 19:33:02.020299 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:02.020311 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:02.020351 (+ 40us) server_negotiation.cc:407] Connection header received
0926 19:33:02.021995 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:02.022015 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:02.022398 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:02.022729 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:02.024504 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:02.025994 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:02.027721 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:02.028368 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:02.028494 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:02.033330 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:02.033342 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:02.033365 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:02.033416 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:02.036535 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:02.037316 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:02.037324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:02.037331 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:02.037490 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:02.037867 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:02.037874 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:02.037877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:02.038441 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:02.038758 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:02.039258 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 19:33:02.039665 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":428,"threads_started":1}
I20260926 19:33:02.049172 7617 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f503740a52482bb585b7846b333267" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42657 } }
I20260926 19:33:02.049588 7617 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:02.049701 7617 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f503740a52482bb585b7846b333267, State: Initialized, Role: FOLLOWER
I20260926 19:33:02.050323 7617 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f503740a52482bb585b7846b333267" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42657 } }
I20260926 19:33:02.050551 7617 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:02.050698 7617 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:02.050887 7617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:02.053050 7617 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f503740a52482bb585b7846b333267" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42657 } }
I20260926 19:33:02.053592 7617 leader_election.cc:302] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8f503740a52482bb585b7846b333267; no voters:
I20260926 19:33:02.054261 7617 leader_election.cc:288] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:02.054477 7622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:02.055963 7622 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [term 1 LEADER]: Becoming Leader. State: Replica: d8f503740a52482bb585b7846b333267, State: Running, Role: LEADER
I20260926 19:33:02.056511 7622 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f503740a52482bb585b7846b333267" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42657 } }
I20260926 19:33:02.056671 7617 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:02.059480 7624 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8f503740a52482bb585b7846b333267. Latest consensus state: current_term: 1 leader_uuid: "d8f503740a52482bb585b7846b333267" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f503740a52482bb585b7846b333267" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42657 } } }
I20260926 19:33:02.059645 7623 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8f503740a52482bb585b7846b333267" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f503740a52482bb585b7846b333267" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42657 } } }
I20260926 19:33:02.060092 7624 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:02.060115 7623 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:02.062949 7631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:02.067983 7631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:02.077735 7631 catalog_manager.cc:1347] Generated new cluster ID: df73d6fb711b4a1c9acdb14681b3b7b5
I20260926 19:33:02.077878 7631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:02.086120 7631 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:02.087940 7631 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:02.097723 7631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8f503740a52482bb585b7846b333267: Generated new TSK 0
I20260926 19:33:02.098661 7631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:02.157117 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42657
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:02.502100 7643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:02.502399 7643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:02.502568 7643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:02.510003 7643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:02.510123 7643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:02.510308 7643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:02.521092 7643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42657
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:02.522455 7643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:02.523890 7643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:02.538403 7648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:02.538362 7649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:02.539011 7643 server_base.cc:1034] running on GCE node
W20260926 19:33:02.538719 7651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:02.539862 7643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:02.541127 7643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:02.542332 7643 hybrid_clock.cc:648] HybridClock initialized: now 1790451182542298 us; error 35 us; skew 500 ppm
I20260926 19:33:02.542662 7643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:02 dist-test-slave-phbh krb5kdc[7537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451182, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:02.546016 7643 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:02.547458 7643 webserver.cc:460] Webserver started at http://127.6.215.193:32813/ using document root <none> and password file <none>
I20260926 19:33:02.548162 7643 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:02.548292 7643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:02.548612 7643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:02.551512 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "c87bf778430448eab4f4734bda4fa9ef"
format_stamp: "Formatted at 2026-09-26 19:33:02 on dist-test-slave-phbh"
server_key: "5d2c212892fc318d38c3164ed77c4e61"
server_key_iv: "dbd0dadb0121738a88a7f6a410e9e637"
server_key_version: "encryptionkey@0"
I20260926 19:33:02.552268 7643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "c87bf778430448eab4f4734bda4fa9ef"
format_stamp: "Formatted at 2026-09-26 19:33:02 on dist-test-slave-phbh"
server_key: "5d2c212892fc318d38c3164ed77c4e61"
server_key_iv: "dbd0dadb0121738a88a7f6a410e9e637"
server_key_version: "encryptionkey@0"
I20260926 19:33:02.557579 7643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 19:33:02.562119 7658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:02.563931 7643 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260926 19:33:02.564097 7643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "c87bf778430448eab4f4734bda4fa9ef"
format_stamp: "Formatted at 2026-09-26 19:33:02 on dist-test-slave-phbh"
server_key: "5d2c212892fc318d38c3164ed77c4e61"
server_key_iv: "dbd0dadb0121738a88a7f6a410e9e637"
server_key_version: "encryptionkey@0"
I20260926 19:33:02.564342 7643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:02.582966 7643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:02.587216 7643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:02.587459 7643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:02.588374 7643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:02.589749 7643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:02.589830 7643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:02.589927 7643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:02.589972 7643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:02.624226 7643 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:33925
I20260926 19:33:02.624267 7770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:33925 every 8 connection(s)
I20260926 19:33:02.625597 7643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:02.627465 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7643
I20260926 19:33:02.627713 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:02.628495 7007 external_mini_cluster.cc:1428] Setting key 77060b02b8d61ba712e93c64fd56644b
Sep 26 19:33:02 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451182, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:33:02.647089 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:02.628186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:37373 (local address 127.6.215.254:42657)
0926 19:33:02.628694 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:02.628699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:02.629891 (+ 1192us) server_negotiation.cc:407] Connection header received
0926 19:33:02.631865 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:02.631870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:02.631956 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:02.632074 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:02.633639 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:02.634408 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:02.635974 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:02.636760 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:02.636925 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:02.640389 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:02.640398 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:02.640403 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:02.640445 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:02.643991 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:02.644882 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:02.644888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:02.644891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:02.645022 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:02.645511 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:02.645515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:02.645517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:02.645764 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:02.645884 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:02.646666 (+ 782us) server_negotiation.cc:299] Negotiation successful
0926 19:33:02.646824 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":300,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
I20260926 19:33:02.648228 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:02.628903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42657 (local address 127.6.215.193:37373)
0926 19:33:02.629724 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:02.629771 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:02.631597 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:02.632305 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:02.632313 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:02.632904 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:02.633477 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:02.633494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:02.634570 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:02.634574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:02.635835 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:02.635847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:02.637001 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:02.637007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:02.637245 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:02.637961 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:02.637989 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:02.640149 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:02.644141 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:02.644147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:02.644161 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:02.644573 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:02.645132 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:02.645135 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:02.645138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:02.645371 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:02.645885 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:02.645891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:02.646241 (+ 350us) client_negotiation.cc:769] Sending connection context
0926 19:33:02.646564 (+ 323us) client_negotiation.cc:240] Negotiation successful
0926 19:33:02.646951 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":328,"threads_started":1}
I20260926 19:33:02.649987 7771 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42657
I20260926 19:33:02.650275 7771 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:02.651043 7771 heartbeater.cc:511] Master 127.6.215.254:42657 requested a full tablet report, sending...
I20260926 19:33:02.653550 7582 ts_manager.cc:194] Registered new tserver with Master: c87bf778430448eab4f4734bda4fa9ef (127.6.215.193:33925)
I20260926 19:33:02.655215 7582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:37373
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:02.672116 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42657
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:02.994387 7777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:02.994709 7777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:02.994876 7777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:03.002416 7777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:03.002579 7777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:03.002766 7777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:03.013010 7777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42657
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:03.014601 7777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:03.016248 7777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:03.030803 7782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:03.030925 7785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:03.030762 7783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:03.032434 7777 server_base.cc:1034] running on GCE node
I20260926 19:33:03.033145 7777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:03.034118 7777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:03.035358 7777 hybrid_clock.cc:648] HybridClock initialized: now 1790451183035285 us; error 63 us; skew 500 ppm
I20260926 19:33:03.035676 7777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:03 dist-test-slave-phbh krb5kdc[7537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451183, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:03.038931 7777 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:03.040290 7777 webserver.cc:460] Webserver started at http://127.6.215.194:32825/ using document root <none> and password file <none>
I20260926 19:33:03.041021 7777 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:03.041122 7777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:03.041370 7777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:03.044516 7777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "c54a5cab6deb447e8854e63ac6aea09f"
format_stamp: "Formatted at 2026-09-26 19:33:03 on dist-test-slave-phbh"
server_key: "a72eccf01bd004a4f050d6b042a397b5"
server_key_iv: "302690c9c41a5d512c0815da63b53eae"
server_key_version: "encryptionkey@0"
I20260926 19:33:03.045320 7777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "c54a5cab6deb447e8854e63ac6aea09f"
format_stamp: "Formatted at 2026-09-26 19:33:03 on dist-test-slave-phbh"
server_key: "a72eccf01bd004a4f050d6b042a397b5"
server_key_iv: "302690c9c41a5d512c0815da63b53eae"
server_key_version: "encryptionkey@0"
I20260926 19:33:03.050794 7777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:03.054595 7792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:03.056032 7777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:33:03.056173 7777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "c54a5cab6deb447e8854e63ac6aea09f"
format_stamp: "Formatted at 2026-09-26 19:33:03 on dist-test-slave-phbh"
server_key: "a72eccf01bd004a4f050d6b042a397b5"
server_key_iv: "302690c9c41a5d512c0815da63b53eae"
server_key_version: "encryptionkey@0"
I20260926 19:33:03.056361 7777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:03.105237 7777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:03.109869 7777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:03.110117 7777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:03.111075 7777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:03.112378 7777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:03.112450 7777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:03.112537 7777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:03.112582 7777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:03.145901 7777 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:37807
I20260926 19:33:03.145987 7904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:37807 every 8 connection(s)
I20260926 19:33:03.147183 7777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:03.149940 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7777
I20260926 19:33:03.150146 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:03.150681 7007 external_mini_cluster.cc:1428] Setting key 8d04e6da31fa2e8eda7afc9a6889bd9f
Sep 26 19:33:03 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451183, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:33:03.170403 7907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.149954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:46993 (local address 127.6.215.254:42657)
0926 19:33:03.151902 (+ 1948us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:03.151908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:03.151929 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:33:03.154278 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:03.154282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:03.154346 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:03.154439 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:03.156670 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.157631 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.159330 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.159741 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.159874 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.163979 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:03.163985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:03.163988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:03.164020 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:03.166896 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:03.167919 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:03.167924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:03.167926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:03.168049 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:03.168592 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:03.168601 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:03.168603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:03.169007 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:03.169139 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:03.169957 (+ 818us) server_negotiation.cc:299] Negotiation successful
0926 19:33:03.170134 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1782,"thread_start_us":1694,"threads_started":1}
I20260926 19:33:03.171453 7908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.150729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42657 (local address 127.6.215.194:46993)
0926 19:33:03.151715 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:03.151790 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:03.153935 (+ 2145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:03.154750 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:03.154765 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:03.155569 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:03.156413 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.156441 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.157816 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.157824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.159184 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.159196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.159966 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.159973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.160196 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.161028 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:03.161054 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:03.163708 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:03.167052 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:03.167061 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:03.167081 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:03.167764 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:03.168172 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:03.168178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:03.168183 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:03.168422 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:03.169169 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:03.169175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:03.169520 (+ 345us) client_negotiation.cc:769] Sending connection context
0926 19:33:03.169846 (+ 326us) client_negotiation.cc:240] Negotiation successful
0926 19:33:03.170234 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":369,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
I20260926 19:33:03.172950 7905 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42657
I20260926 19:33:03.173259 7905 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:03.173974 7905 heartbeater.cc:511] Master 127.6.215.254:42657 requested a full tablet report, sending...
I20260926 19:33:03.175519 7582 ts_manager.cc:194] Registered new tserver with Master: c54a5cab6deb447e8854e63ac6aea09f (127.6.215.194:37807)
I20260926 19:33:03.176365 7582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:46993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:03.196050 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42657
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:03.518867 7911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:03.519155 7911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:03.519335 7911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:03.526923 7911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:03.527036 7911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:03.527210 7911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:03.537025 7911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42657
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:03.538319 7911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:03.539624 7911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:03.552654 7916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:03.552613 7917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:03.553053 7919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:03.553520 7911 server_base.cc:1034] running on GCE node
I20260926 19:33:03.554137 7911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:03.555086 7911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:03.556339 7911 hybrid_clock.cc:648] HybridClock initialized: now 1790451183556241 us; error 55 us; skew 500 ppm
I20260926 19:33:03.556655 7911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:03 dist-test-slave-phbh krb5kdc[7537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451183, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:03.559832 7911 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:03.561395 7911 webserver.cc:460] Webserver started at http://127.6.215.195:43703/ using document root <none> and password file <none>
I20260926 19:33:03.562114 7911 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:03.562209 7911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:03.562464 7911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:03.565340 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "ca535a4642934f0b92229c01c0ea2ba5"
format_stamp: "Formatted at 2026-09-26 19:33:03 on dist-test-slave-phbh"
server_key: "f75d16e59af5b38431297ffb7acd9824"
server_key_iv: "99808a5b2ffff30f802da01d01d76492"
server_key_version: "encryptionkey@0"
I20260926 19:33:03.566020 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "ca535a4642934f0b92229c01c0ea2ba5"
format_stamp: "Formatted at 2026-09-26 19:33:03 on dist-test-slave-phbh"
server_key: "f75d16e59af5b38431297ffb7acd9824"
server_key_iv: "99808a5b2ffff30f802da01d01d76492"
server_key_version: "encryptionkey@0"
I20260926 19:33:03.570927 7911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:03.574385 7926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:03.575663 7911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:33:03.575819 7911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "ca535a4642934f0b92229c01c0ea2ba5"
format_stamp: "Formatted at 2026-09-26 19:33:03 on dist-test-slave-phbh"
server_key: "f75d16e59af5b38431297ffb7acd9824"
server_key_iv: "99808a5b2ffff30f802da01d01d76492"
server_key_version: "encryptionkey@0"
I20260926 19:33:03.576020 7911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:03.587468 7911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:03.591638 7911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:03.591961 7911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:03.593034 7911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:03.594565 7911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:03.594669 7911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:03.594794 7911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:03.594869 7911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:03.633057 7911 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:33173
I20260926 19:33:03.633091 8038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:33173 every 8 connection(s)
I20260926 19:33:03.634287 7911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:03.643172 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 7911
I20260926 19:33:03.643431 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:03.643860 7007 external_mini_cluster.cc:1428] Setting key dd773ccfb0df99ae1b0355d150e7b20e
Sep 26 19:33:03 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451183, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:03.653371 7907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.636832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:39023 (local address 127.6.215.254:42657)
0926 19:33:03.637037 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:03.637043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:03.638142 (+ 1099us) server_negotiation.cc:407] Connection header received
0926 19:33:03.639839 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:03.639843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:03.639912 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:03.640001 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:03.641469 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.642200 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.643766 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.644230 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.644377 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.647918 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:03.647923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:03.647926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:03.647956 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:03.650479 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:03.651174 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:03.651178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:03.651180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:03.651346 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:03.651705 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:03.651709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:03.651711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:03.651962 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:03.652072 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:03.652966 (+ 894us) server_negotiation.cc:299] Negotiation successful
0926 19:33:03.653135 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:33:03.654484 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.637324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42657 (local address 127.6.215.195:39023)
0926 19:33:03.637992 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:03.638026 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:03.639585 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:03.640189 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:03.640197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:03.640677 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:03.641312 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.641329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.642325 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.642328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.643617 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.643633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.644461 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.644468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.644708 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.645436 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:03.645457 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:03.647743 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:03.650617 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:03.650624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:03.650636 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:03.651062 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:03.651444 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:03.651447 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:03.651449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:03.651610 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:03.652126 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:03.652132 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:03.652549 (+ 417us) client_negotiation.cc:769] Sending connection context
0926 19:33:03.652888 (+ 339us) client_negotiation.cc:240] Negotiation successful
0926 19:33:03.653351 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":252,"threads_started":1}
I20260926 19:33:03.655869 8039 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42657
I20260926 19:33:03.656185 8039 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:03.656975 8039 heartbeater.cc:511] Master 127.6.215.254:42657 requested a full tablet report, sending...
I20260926 19:33:03.658278 7771 heartbeater.cc:503] Master 127.6.215.254:42657 was elected leader, sending a full tablet report...
I20260926 19:33:03.658609 7582 ts_manager.cc:194] Registered new tserver with Master: ca535a4642934f0b92229c01c0ea2ba5 (127.6.215.195:33173)
I20260926 19:33:03.659173 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:33:03.659670 7582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:39023
I20260926 19:33:03.679052 7907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.666187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.6.215.254:42657)
0926 19:33:03.666371 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:03.666378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:03.666916 (+ 538us) server_negotiation.cc:407] Connection header received
0926 19:33:03.667050 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:03.667053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:03.667108 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:03.667186 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:03.668686 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.669459 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.670839 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.671217 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.671308 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.673828 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:03.673837 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:03.673842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:03.673882 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:03.676606 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:03.677279 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:03.677283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:03.677286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:03.677409 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:03.677712 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:03.677715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:03.677717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:03.678015 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:03.678198 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:03.678532 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 19:33:03.678677 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 19:33:03.685521 7582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:03.692118 7582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:03.724102 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.703681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37807 (local address 127.0.0.1:38222)
0926 19:33:03.713797 (+ 10116us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:03.713850 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:03.714084 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:03.715699 (+ 1615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:03.715707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:03.715749 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:03.716171 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.716189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.718195 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.718200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.721045 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.721062 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.722946 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.722951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.723171 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.723222 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 19:33:03.723354 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 19:33:03.723558 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9781,"mutex_wait_us":117,"thread_start_us":11108,"threads_started":1}
I20260926 19:33:03.724311 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.704088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:33173 (local address 127.0.0.1:60296)
0926 19:33:03.715051 (+ 10963us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:03.715068 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:03.715189 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:03.716007 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:03.716014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:03.716044 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:03.716491 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.716504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.718415 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.718419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.721520 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.721535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.723317 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.723324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.723512 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.723545 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 19:33:03.723642 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 19:33:03.723812 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10849,"mutex_wait_us":562,"thread_start_us":9788,"threads_started":1}
I20260926 19:33:03.724429 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.703901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.6.215.194:37807)
0926 19:33:03.714806 (+ 10905us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:03.714815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:03.714842 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:33:03.714945 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:03.714952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:03.715214 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:03.715413 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:03.716389 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.718032 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.721241 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.722711 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.722864 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.723058 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:03.723929 (+ 871us) server_negotiation.cc:299] Negotiation successful
0926 19:33:03.724109 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10697,"thread_start_us":10558,"threads_started":1}
I20260926 19:33:03.725090 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.705104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.6.215.195:33173)
0926 19:33:03.713217 (+ 8113us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:03.713226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:03.715180 (+ 1954us) server_negotiation.cc:407] Connection header received
0926 19:33:03.715325 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:03.715335 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:03.715562 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:03.715753 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:03.716704 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.718222 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.721694 (+ 3472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.723090 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.723259 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.723493 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:03.724533 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0926 19:33:03.724713 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7933,"thread_start_us":7771,"threads_started":1}
I20260926 19:33:03.728403 7974 tablet_service.cc:1467] Processing CreateTablet for tablet 49b9a0936f7c425780422cf01a054489 (DEFAULT_TABLE table=test-table [id=81777b7cbfbd4171bb5a70a39c1b13f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:03.728355 7840 tablet_service.cc:1467] Processing CreateTablet for tablet 49b9a0936f7c425780422cf01a054489 (DEFAULT_TABLE table=test-table [id=81777b7cbfbd4171bb5a70a39c1b13f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:03.729684 7840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b9a0936f7c425780422cf01a054489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:03.729781 7974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b9a0936f7c425780422cf01a054489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:03.731930 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.715438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33925 (local address 127.0.0.1:38276)
0926 19:33:03.722762 (+ 7324us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:03.722776 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:03.722933 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:03.723634 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:03.723638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:03.723664 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:03.724052 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.724063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.727150 (+ 3087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.727156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.729082 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:03.729096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.731145 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.731152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:03.731335 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.731377 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 19:33:03.731476 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 19:33:03.731606 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7212,"thread_start_us":7078,"threads_started":1}
I20260926 19:33:03.732040 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:03.715071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.6.215.193:33925)
0926 19:33:03.721353 (+ 6282us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:03.721363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:03.723064 (+ 1701us) server_negotiation.cc:407] Connection header received
0926 19:33:03.723172 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:03.723181 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:03.723434 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:03.723630 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:03.724558 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.726990 (+ 2432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.729257 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:03.730640 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:03.730783 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:03.730998 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:03.731550 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 19:33:03.731707 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6084,"thread_start_us":6315,"threads_started":1}
I20260926 19:33:03.735999 7706 tablet_service.cc:1467] Processing CreateTablet for tablet 49b9a0936f7c425780422cf01a054489 (DEFAULT_TABLE table=test-table [id=81777b7cbfbd4171bb5a70a39c1b13f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:03.737596 7706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b9a0936f7c425780422cf01a054489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:03.741365 8057 tablet_bootstrap.cc:492] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5: Bootstrap starting.
I20260926 19:33:03.743438 8058 tablet_bootstrap.cc:492] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f: Bootstrap starting.
I20260926 19:33:03.745775 8057 tablet_bootstrap.cc:654] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:03.746919 8058 tablet_bootstrap.cc:654] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:03.747211 8057 log.cc:824] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:03.748124 8058 log.cc:824] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:03.750866 8057 tablet_bootstrap.cc:492] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5: No bootstrap required, opened a new log
I20260926 19:33:03.751188 8057 ts_tablet_manager.cc:1397] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260926 19:33:03.751482 8061 tablet_bootstrap.cc:492] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef: Bootstrap starting.
I20260926 19:33:03.755056 8061 tablet_bootstrap.cc:654] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:03.755455 8058 tablet_bootstrap.cc:492] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f: No bootstrap required, opened a new log
I20260926 19:33:03.755710 8058 ts_tablet_manager.cc:1397] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260926 19:33:03.756155 8061 log.cc:824] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:03.758793 8061 tablet_bootstrap.cc:492] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef: No bootstrap required, opened a new log
I20260926 19:33:03.759037 8061 ts_tablet_manager.cc:1397] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260926 19:33:03.763053 8057 raft_consensus.cc:348] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:03.764197 8057 raft_consensus.cc:374] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:03.764305 8057 raft_consensus.cc:729] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca535a4642934f0b92229c01c0ea2ba5, State: Initialized, Role: FOLLOWER
I20260926 19:33:03.764981 8057 consensus_queue.cc:260] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:03.766047 8039 heartbeater.cc:503] Master 127.6.215.254:42657 was elected leader, sending a full tablet report...
I20260926 19:33:03.766088 8058 raft_consensus.cc:348] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:03.766520 8058 raft_consensus.cc:374] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:03.766630 8058 raft_consensus.cc:729] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c54a5cab6deb447e8854e63ac6aea09f, State: Initialized, Role: FOLLOWER
I20260926 19:33:03.767175 8058 consensus_queue.cc:260] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:03.768234 8061 raft_consensus.cc:348] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:03.768635 8061 raft_consensus.cc:374] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:03.768746 8061 raft_consensus.cc:729] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87bf778430448eab4f4734bda4fa9ef, State: Initialized, Role: FOLLOWER
I20260926 19:33:03.769333 8061 consensus_queue.cc:260] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:03.771864 8061 ts_tablet_manager.cc:1428] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260926 19:33:03.772209 8057 ts_tablet_manager.cc:1428] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5: Time spent starting tablet: real 0.021s user 0.008s sys 0.006s
I20260926 19:33:03.773111 7905 heartbeater.cc:503] Master 127.6.215.254:42657 was elected leader, sending a full tablet report...
I20260926 19:33:03.773773 8058 ts_tablet_manager.cc:1428] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f: Time spent starting tablet: real 0.018s user 0.016s sys 0.003s
W20260926 19:33:03.878033 7772 tablet.cc:2369] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:33:03.885852 8040 tablet.cc:2369] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:33:03.899428 7906 tablet.cc:2369] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:04.041352 8064 raft_consensus.cc:482] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:04.041642 8064 raft_consensus.cc:504] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:04.043200 8064 leader_election.cc:288] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca535a4642934f0b92229c01c0ea2ba5 (127.6.215.195:33173), c87bf778430448eab4f4734bda4fa9ef (127.6.215.193:33925)
I20260926 19:33:04.051887 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.044546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:54819 (local address 127.6.215.193:33925)
0926 19:33:04.044736 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.044745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.045249 (+ 504us) server_negotiation.cc:407] Connection header received
0926 19:33:04.045423 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.045427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.045519 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.045625 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:04.046059 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.047142 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.049627 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.050894 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.051039 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.051098 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.051472 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.051611 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 19:33:04.051880 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.043751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:53797 (local address 127.6.215.195:33173)
0926 19:33:04.044032 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.044039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.044399 (+ 360us) server_negotiation.cc:407] Connection header received
0926 19:33:04.044622 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.044631 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.044747 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.044895 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:04.045557 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.046822 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.049484 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.050724 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.050905 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.050999 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.051457 (+ 458us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.051607 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 19:33:04.051967 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.043628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:33173 (local address 127.6.215.194:53797)
0926 19:33:04.044237 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:04.044253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:04.044444 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:04.044923 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:04.044931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:04.044967 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:04.045412 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:04.045426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.046965 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.046969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:04.049269 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:04.049284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.050982 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.050991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:04.051199 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.051241 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 19:33:04.051343 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 19:33:04.051493 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":349,"threads_started":1}
I20260926 19:33:04.051988 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.044393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33925 (local address 127.6.215.194:54819)
0926 19:33:04.045129 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:04.045147 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:04.045287 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:04.045645 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:04.045648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:04.045667 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:04.045912 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:04.045918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.047291 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.047296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:04.049454 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:04.049467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.051089 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.051095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:04.051263 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.051312 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 19:33:04.051390 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 19:33:04.051492 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":5,"thread_start_us":395,"threads_started":1}
I20260926 19:33:04.053309 7994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b9a0936f7c425780422cf01a054489" candidate_uuid: "c54a5cab6deb447e8854e63ac6aea09f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca535a4642934f0b92229c01c0ea2ba5" is_pre_election: true
I20260926 19:33:04.053479 7726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b9a0936f7c425780422cf01a054489" candidate_uuid: "c54a5cab6deb447e8854e63ac6aea09f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87bf778430448eab4f4734bda4fa9ef" is_pre_election: true
I20260926 19:33:04.053853 7994 raft_consensus.cc:2454] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c54a5cab6deb447e8854e63ac6aea09f in term 0.
I20260926 19:33:04.054059 7726 raft_consensus.cc:2454] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c54a5cab6deb447e8854e63ac6aea09f in term 0.
I20260926 19:33:04.054639 7796 leader_election.cc:302] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c54a5cab6deb447e8854e63ac6aea09f, ca535a4642934f0b92229c01c0ea2ba5; no voters:
I20260926 19:33:04.055038 8064 raft_consensus.cc:2781] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:04.055182 8064 raft_consensus.cc:482] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:04.055316 8064 raft_consensus.cc:3037] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:04.057632 8064 raft_consensus.cc:504] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:04.058277 8064 leader_election.cc:288] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [CANDIDATE]: Term 1 election: Requested vote from peers ca535a4642934f0b92229c01c0ea2ba5 (127.6.215.195:33173), c87bf778430448eab4f4734bda4fa9ef (127.6.215.193:33925)
I20260926 19:33:04.058758 7994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b9a0936f7c425780422cf01a054489" candidate_uuid: "c54a5cab6deb447e8854e63ac6aea09f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca535a4642934f0b92229c01c0ea2ba5"
I20260926 19:33:04.058784 7726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b9a0936f7c425780422cf01a054489" candidate_uuid: "c54a5cab6deb447e8854e63ac6aea09f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87bf778430448eab4f4734bda4fa9ef"
I20260926 19:33:04.058945 7994 raft_consensus.cc:3037] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:04.059010 7726 raft_consensus.cc:3037] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:04.061502 7994 raft_consensus.cc:2454] T 49b9a0936f7c425780422cf01a054489 P ca535a4642934f0b92229c01c0ea2ba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c54a5cab6deb447e8854e63ac6aea09f in term 1.
I20260926 19:33:04.062031 7726 raft_consensus.cc:2454] T 49b9a0936f7c425780422cf01a054489 P c87bf778430448eab4f4734bda4fa9ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c54a5cab6deb447e8854e63ac6aea09f in term 1.
I20260926 19:33:04.062057 7796 leader_election.cc:302] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c54a5cab6deb447e8854e63ac6aea09f, ca535a4642934f0b92229c01c0ea2ba5; no voters:
I20260926 19:33:04.062445 8064 raft_consensus.cc:2781] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:04.062925 8064 raft_consensus.cc:686] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [term 1 LEADER]: Becoming Leader. State: Replica: c54a5cab6deb447e8854e63ac6aea09f, State: Running, Role: LEADER
I20260926 19:33:04.063418 8064 consensus_queue.cc:237] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } }
I20260926 19:33:04.068620 7582 catalog_manager.cc:5495] T 49b9a0936f7c425780422cf01a054489 P c54a5cab6deb447e8854e63ac6aea09f reported cstate change: term changed from 0 to 1, leader changed from <none> to c54a5cab6deb447e8854e63ac6aea09f (127.6.215.194). New cstate: current_term: 1 leader_uuid: "c54a5cab6deb447e8854e63ac6aea09f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c54a5cab6deb447e8854e63ac6aea09f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca535a4642934f0b92229c01c0ea2ba5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c87bf778430448eab4f4734bda4fa9ef" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33925 } health_report { overall_health: UNKNOWN } } }
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:04.095765 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.080602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.6.215.193:33925)
0926 19:33:04.080904 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.080914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.081345 (+ 431us) server_negotiation.cc:407] Connection header received
0926 19:33:04.081502 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.081506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.081567 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.081648 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:04.083228 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.083999 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.085463 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.085861 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.085962 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.090110 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:04.090124 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:04.090139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:04.090179 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:04.093100 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.093751 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.093758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.093760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.093885 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.094255 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.094265 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.094275 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.094597 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:04.094889 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.095201 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.095421 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[7537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451184, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451184, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:04.140427 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.126712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38296 (local address 127.6.215.193:33925)
0926 19:33:04.126927 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.126935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.127263 (+ 328us) server_negotiation.cc:407] Connection header received
0926 19:33:04.127482 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.127487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.127548 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.127653 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:04.129019 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.129901 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.131199 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.131633 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.131736 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.135533 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:04.135539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:04.135542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:04.135574 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:04.138280 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.138821 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.138825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.138827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.138949 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.139279 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.139283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.139285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.139561 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:04.139775 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.140003 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.140167 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[7537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:04.186645 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.172530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38306 (local address 127.6.215.193:33925)
0926 19:33:04.172745 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.172751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.172994 (+ 243us) server_negotiation.cc:407] Connection header received
0926 19:33:04.173142 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.173146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.173208 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.173293 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:04.174634 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.175340 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.176530 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.177031 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.177133 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.181513 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:04.181518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:04.181522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:04.181557 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:04.184111 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.184738 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.184742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.184744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.184954 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.185322 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.185327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.185329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.185677 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:04.185896 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.186131 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.186373 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260926 19:33:04.188500 7686 tablet_service.cc:2908] Rejecting scan request for tablet 49b9a0936f7c425780422cf01a054489: Uninitialized: clean time has not yet been initialized
I20260926 19:33:04.191634 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7643
I20260926 19:33:04.204193 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.192510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38314 (local address 127.6.215.193:33925)
0926 19:33:04.192669 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.192673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.193045 (+ 372us) server_negotiation.cc:407] Connection header received
0926 19:33:04.193163 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.193167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.193233 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.193320 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:04.194448 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.195131 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.196246 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.196694 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.196925 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.199053 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:04.199059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:04.199062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:04.199093 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:04.201926 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.202534 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.202538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.202540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.202653 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.202989 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.202993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.202996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.203331 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:04.203559 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.203776 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.203946 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:33:04.204943 7766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:04.297026 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7643
I20260926 19:33:04.313256 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7777
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:33:04.327808 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.314034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38230 (local address 127.6.215.194:37807)
0926 19:33:04.314568 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.314573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.314588 (+ 15us) server_negotiation.cc:407] Connection header received
0926 19:33:04.314638 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.314642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.314703 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.314792 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:04.315924 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.316612 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.317965 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.318377 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.318470 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.322506 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:04.322521 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:04.322535 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:04.322569 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:04.325393 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.325998 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.326006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.326008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.326125 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.326505 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.326510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.326512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.326863 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:04.327102 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.327366 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.327530 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":332,"threads_started":1}
I20260926 19:33:04.328464 7900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:04.417802 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7777
I20260926 19:33:04.435402 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7911
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[7537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:04.451045 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:04.436158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.6.215.195:33173)
0926 19:33:04.436341 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:04.436349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:04.436410 (+ 61us) server_negotiation.cc:407] Connection header received
0926 19:33:04.436557 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:04.436561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:04.436624 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:04.436707 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:04.437955 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.438695 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.440056 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:04.440423 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:04.440519 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:04.444966 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:04.444981 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:04.444997 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:04.445038 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:04.448099 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.449011 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.449019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.449021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.449158 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:04.449659 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:04.449663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:04.449665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:04.450021 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:04.450300 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:04.450525 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 19:33:04.450683 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:33:04.451735 8034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:04.538333 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7911
I20260926 19:33:04.555490 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 7550
I20260926 19:33:04.556887 7612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:04.663338 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 7550
2026-09-26T19:33:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4357 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[8088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[8088](info): set up 2 sockets
Sep 26 19:33:04 dist-test-slave-phbh krb5kdc[8088](info): commencing operation
krb5kdc: starting...
W20260926 19:33:05.754114 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:05 dist-test-slave-phbh krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:05.936869 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42831
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:06.255327 8101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:06.255661 8101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:06.255829 8101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:06.262910 8101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:06.263058 8101 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 19:33:06.263136 8101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:06.263228 8101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:06.263337 8101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:06.263429 8101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:06.273284 8101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42831
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:06.274566 8101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:06.276242 8101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:06.288143 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:06.288230 8107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:06.288182 8109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:06.289149 8101 server_base.cc:1034] running on GCE node
I20260926 19:33:06.289968 8101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:06.291558 8101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:06.292878 8101 hybrid_clock.cc:648] HybridClock initialized: now 1790451186292781 us; error 58 us; skew 500 ppm
I20260926 19:33:06.293305 8101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:06 dist-test-slave-phbh krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451186, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:06.296857 8101 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:06.298457 8101 webserver.cc:460] Webserver started at http://127.6.215.254:43295/ using document root <none> and password file <none>
I20260926 19:33:06.299247 8101 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:06.299379 8101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:06.299728 8101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:06.303310 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "8e31334c16a74ea39d6268ca9697d6cc"
format_stamp: "Formatted at 2026-09-26 19:33:06 on dist-test-slave-phbh"
server_key: "2f2e0a4107b58de574f14c7f3f0a0576"
server_key_iv: "1fbda240e94e2f6912c8c296c92407e1"
server_key_version: "encryptionkey@0"
I20260926 19:33:06.304150 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "8e31334c16a74ea39d6268ca9697d6cc"
format_stamp: "Formatted at 2026-09-26 19:33:06 on dist-test-slave-phbh"
server_key: "2f2e0a4107b58de574f14c7f3f0a0576"
server_key_iv: "1fbda240e94e2f6912c8c296c92407e1"
server_key_version: "encryptionkey@0"
I20260926 19:33:06.309471 8101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 19:33:06.312940 8116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:06.314337 8101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:33:06.314525 8101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "8e31334c16a74ea39d6268ca9697d6cc"
format_stamp: "Formatted at 2026-09-26 19:33:06 on dist-test-slave-phbh"
server_key: "2f2e0a4107b58de574f14c7f3f0a0576"
server_key_iv: "1fbda240e94e2f6912c8c296c92407e1"
server_key_version: "encryptionkey@0"
I20260926 19:33:06.314744 8101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:06.340639 8101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:06.345458 8101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:06.345745 8101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:06.368438 8101 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42831
I20260926 19:33:06.368456 8177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42831 every 8 connection(s)
I20260926 19:33:06.369899 8101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:06.373750 8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:06.374176 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 8101
I20260926 19:33:06.374436 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:06.375082 7007 external_mini_cluster.cc:1428] Setting key 0504206b2d9fa7cf5edb665515202f5c
I20260926 19:33:06.388015 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc: Bootstrap starting.
Sep 26 19:33:06 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:06.391723 8178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:06.392967 8178 log.cc:824] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:06.396169 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc: No bootstrap required, opened a new log
I20260926 19:33:06.398666 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:06.377213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54262 (local address 127.6.215.254:42831)
0926 19:33:06.378043 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:06.378055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:06.378094 (+ 39us) server_negotiation.cc:407] Connection header received
0926 19:33:06.379726 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:06.379751 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:06.380158 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:06.380505 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:06.382299 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.383583 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.385409 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.386101 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.386234 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:06.391135 (+ 4901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:06.391143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:06.391160 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:06.391199 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:06.394611 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:06.395337 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:06.395343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:06.395350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:06.395520 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:06.395916 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:06.395920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:06.395922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:06.396482 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:06.396814 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:06.397248 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 19:33:06.397664 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":392,"threads_started":1}
I20260926 19:33:06.405755 8178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31334c16a74ea39d6268ca9697d6cc" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42831 } }
I20260926 19:33:06.406149 8178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:06.406265 8178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e31334c16a74ea39d6268ca9697d6cc, State: Initialized, Role: FOLLOWER
I20260926 19:33:06.406973 8178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31334c16a74ea39d6268ca9697d6cc" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42831 } }
I20260926 19:33:06.407225 8178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:06.407420 8178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:06.407617 8178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:06.410125 8178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31334c16a74ea39d6268ca9697d6cc" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42831 } }
I20260926 19:33:06.410620 8178 leader_election.cc:302] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e31334c16a74ea39d6268ca9697d6cc; no voters:
I20260926 19:33:06.411216 8178 leader_election.cc:288] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:06.411423 8183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:06.412137 8183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [term 1 LEADER]: Becoming Leader. State: Replica: 8e31334c16a74ea39d6268ca9697d6cc, State: Running, Role: LEADER
I20260926 19:33:06.412726 8183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31334c16a74ea39d6268ca9697d6cc" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42831 } }
I20260926 19:33:06.413293 8178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:06.415552 8184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e31334c16a74ea39d6268ca9697d6cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31334c16a74ea39d6268ca9697d6cc" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42831 } } }
I20260926 19:33:06.415951 8184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:06.416298 8185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e31334c16a74ea39d6268ca9697d6cc. Latest consensus state: current_term: 1 leader_uuid: "8e31334c16a74ea39d6268ca9697d6cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e31334c16a74ea39d6268ca9697d6cc" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42831 } } }
I20260926 19:33:06.416592 8185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:06.418418 8190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:06.423406 8190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:06.432658 8190 catalog_manager.cc:1347] Generated new cluster ID: d832659ea05d48b583e94b225b89cf95
I20260926 19:33:06.432821 8190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:06.470013 8190 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:06.471575 8190 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:06.486392 8190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc: Generated new TSK 0
I20260926 19:33:06.487465 8190 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:06.545200 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42831
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:06.871711 8204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:06.872165 8204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:06.872360 8204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:06.879719 8204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:06.879874 8204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:06.880043 8204 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:06.880188 8204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:06.889973 8204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:42831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:06.891366 8204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:06.892690 8204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:06.906365 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:06.906381 8212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:06.906425 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:06.907248 8204 server_base.cc:1034] running on GCE node
I20260926 19:33:06.907842 8204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:06.908804 8204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:06.910060 8204 hybrid_clock.cc:648] HybridClock initialized: now 1790451186909990 us; error 46 us; skew 500 ppm
I20260926 19:33:06.910396 8204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:06 dist-test-slave-phbh krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451186, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:06.913661 8204 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:06.915125 8204 webserver.cc:460] Webserver started at http://127.6.215.193:42977/ using document root <none> and password file <none>
I20260926 19:33:06.915812 8204 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:06.915921 8204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:06.916244 8204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:06.919418 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "9a4ec3420a674fbea8676175121989d1"
format_stamp: "Formatted at 2026-09-26 19:33:06 on dist-test-slave-phbh"
server_key: "eb2870602dea1d334cfee7c1a4ca573b"
server_key_iv: "9f6522112241b53adfdcb06f5db5007e"
server_key_version: "encryptionkey@0"
I20260926 19:33:06.920208 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "9a4ec3420a674fbea8676175121989d1"
format_stamp: "Formatted at 2026-09-26 19:33:06 on dist-test-slave-phbh"
server_key: "eb2870602dea1d334cfee7c1a4ca573b"
server_key_iv: "9f6522112241b53adfdcb06f5db5007e"
server_key_version: "encryptionkey@0"
I20260926 19:33:06.925271 8204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 19:33:06.928756 8219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:06.930174 8204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:33:06.930341 8204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "9a4ec3420a674fbea8676175121989d1"
format_stamp: "Formatted at 2026-09-26 19:33:06 on dist-test-slave-phbh"
server_key: "eb2870602dea1d334cfee7c1a4ca573b"
server_key_iv: "9f6522112241b53adfdcb06f5db5007e"
server_key_version: "encryptionkey@0"
I20260926 19:33:06.930568 8204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:06.951366 8204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:06.955626 8204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:06.955924 8204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:06.958420 8204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:06.958535 8204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:06.958680 8204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:06.958758 8204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:06 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451186, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:06.984089 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:06.959849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42831 (local address 127.6.215.193:35265)
0926 19:33:06.960715 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:06.960763 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:06.962696 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:06.963759 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:06.963779 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:06.964746 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:06.965895 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:06.965929 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.967285 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.967292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:06.970289 (+ 2997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:06.970307 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.971388 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.971399 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:06.971754 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:06.972660 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:06.972684 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:06.975914 (+ 3230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:06.979825 (+ 3911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:06.979834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:06.979851 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:06.980257 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:06.980629 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:06.980634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:06.980638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:06.980929 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:06.981595 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:06.981603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:06.982128 (+ 525us) client_negotiation.cc:769] Sending connection context
0926 19:33:06.982432 (+ 304us) client_negotiation.cc:240] Negotiation successful
0926 19:33:06.982965 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":381,"threads_started":1}
I20260926 19:33:06.985234 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:06.959218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:35265 (local address 127.6.215.254:42831)
0926 19:33:06.959694 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:06.959701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:06.960891 (+ 1190us) server_negotiation.cc:407] Connection header received
0926 19:33:06.962993 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:06.962999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:06.963080 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:06.963190 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:06.966136 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.967090 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.970540 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.971072 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.971215 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:06.976224 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:06.976233 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:06.976240 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:06.976288 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:06.979675 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:06.980390 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:06.980396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:06.980400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:06.980518 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:06.981049 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:06.981054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:06.981058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:06.981480 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:06.981619 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:06.984613 (+ 2994us) server_negotiation.cc:299] Negotiation successful
0926 19:33:06.984871 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":250,"threads_started":1}
I20260926 19:33:07.005298 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:06.990979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:40843 (local address 127.6.215.254:42831)
0926 19:33:06.991197 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:06.991205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:06.991719 (+ 514us) server_negotiation.cc:407] Connection header received
0926 19:33:06.991858 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:06.991867 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:06.991963 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:06.992102 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:06.993033 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.993909 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.995964 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.996365 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.996503 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:06.998581 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:06.998591 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:06.998597 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:06.998642 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:07.002155 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:07.003208 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:07.003216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:07.003222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:07.003395 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:07.003880 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:07.003883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:07.003885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:07.004129 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:07.004331 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:07.004684 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 19:33:07.004939 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 19:33:07.005439 8306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:06.990994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42831 (local address 127.6.215.193:40843)
0926 19:33:06.991622 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:06.991638 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:06.991745 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:06.992112 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:06.992115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:06.992538 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:06.992900 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:06.992910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.994090 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.994094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:06.995829 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:06.995840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:06.996544 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:06.996549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:06.996707 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:06.997396 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:06.997413 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:06.998399 (+ 986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:07.002329 (+ 3930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:07.002341 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:07.002348 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:07.003072 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:07.003552 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:07.003560 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:07.003566 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:07.003758 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:07.004251 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:07.004256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:07.004450 (+ 194us) client_negotiation.cc:769] Sending connection context
0926 19:33:07.004594 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 19:33:07.004986 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":347,"threads_started":1}
I20260926 19:33:07.010860 8204 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:43043
I20260926 19:33:07.010895 8339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:43043 every 8 connection(s)
I20260926 19:33:07.011969 8204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:07.013944 8340 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42831
I20260926 19:33:07.014189 8340 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:07.014704 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 8204
I20260926 19:33:07.014918 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:07.014941 8340 heartbeater.cc:511] Master 127.6.215.254:42831 requested a full tablet report, sending...
I20260926 19:33:07.015520 7007 external_mini_cluster.cc:1428] Setting key c1025a4a07c0371966d4cdeb8ee07d11
I20260926 19:33:07.016997 8133 ts_manager.cc:194] Registered new tserver with Master: 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043)
I20260926 19:33:07.018690 8133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:35265
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:07.056968 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42831
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:07.379388 8344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:07.379756 8344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:07.379899 8344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:07.388319 8344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:07.388479 8344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:07.388620 8344 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:07.388752 8344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:07.398834 8344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:42831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:07.400146 8344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:07.401600 8344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:07.417601 8350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:07.417531 8349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:07.418215 8352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:07.420519 8344 server_base.cc:1034] running on GCE node
I20260926 19:33:07.421236 8344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:07.422284 8344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:07.423581 8344 hybrid_clock.cc:648] HybridClock initialized: now 1790451187423497 us; error 47 us; skew 500 ppm
I20260926 19:33:07.423949 8344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:07 dist-test-slave-phbh krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451187, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:07.427399 8344 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:07.429095 8344 webserver.cc:460] Webserver started at http://127.6.215.194:37653/ using document root <none> and password file <none>
I20260926 19:33:07.429875 8344 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:07.430011 8344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:07.430361 8344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:07.434517 8344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "16c411e316f94a579fd5b838c68fff7b"
format_stamp: "Formatted at 2026-09-26 19:33:07 on dist-test-slave-phbh"
server_key: "193e5f6a8cd84f2838af56fd21c464e8"
server_key_iv: "002d0c78c4e3f419b9316423f568aedd"
server_key_version: "encryptionkey@0"
I20260926 19:33:07.435598 8344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "16c411e316f94a579fd5b838c68fff7b"
format_stamp: "Formatted at 2026-09-26 19:33:07 on dist-test-slave-phbh"
server_key: "193e5f6a8cd84f2838af56fd21c464e8"
server_key_iv: "002d0c78c4e3f419b9316423f568aedd"
server_key_version: "encryptionkey@0"
I20260926 19:33:07.440836 8344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 19:33:07.444322 8359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:07.445739 8344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:33:07.445915 8344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "16c411e316f94a579fd5b838c68fff7b"
format_stamp: "Formatted at 2026-09-26 19:33:07 on dist-test-slave-phbh"
server_key: "193e5f6a8cd84f2838af56fd21c464e8"
server_key_iv: "002d0c78c4e3f419b9316423f568aedd"
server_key_version: "encryptionkey@0"
I20260926 19:33:07.446153 8344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:07.463032 8344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:07.467254 8344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:07.467535 8344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:07.470459 8344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:07.470575 8344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:07.470714 8344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:07.470786 8344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:07 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451187, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:07.498801 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:07.471542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42831 (local address 127.6.215.194:43117)
0926 19:33:07.472384 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:07.472426 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:07.474482 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:07.475276 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:07.475285 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:07.476030 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:07.476987 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:07.477019 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.478495 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.478502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:07.480854 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:07.480867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.481689 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.481697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:07.481924 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:07.482724 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:07.482751 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:07.486458 (+ 3707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:07.491959 (+ 5501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:07.492007 (+ 48us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:07.492035 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:07.492978 (+ 943us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:07.493586 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:07.493594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:07.493601 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:07.493959 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:07.495035 (+ 1076us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:07.495047 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:07.495675 (+ 628us) client_negotiation.cc:769] Sending connection context
0926 19:33:07.496080 (+ 405us) client_negotiation.cc:240] Negotiation successful
0926 19:33:07.496854 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":389,"threads_started":1}
I20260926 19:33:07.500624 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:07.470821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:43117 (local address 127.6.215.254:42831)
0926 19:33:07.471018 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:07.471025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:07.472553 (+ 1528us) server_negotiation.cc:407] Connection header received
0926 19:33:07.474775 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:07.474781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:07.474908 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:07.475050 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:07.477252 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.478326 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.480999 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.481451 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.481584 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:07.486834 (+ 5250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:07.486844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:07.486853 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:07.486921 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:07.491660 (+ 4739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:07.493224 (+ 1564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:07.493229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:07.493233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:07.493408 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:07.494248 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:07.494256 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:07.494261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:07.494732 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:07.494944 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:07.499885 (+ 4941us) server_negotiation.cc:299] Negotiation successful
0926 19:33:07.500178 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:33:07.539067 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:07.523620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:57909 (local address 127.6.215.254:42831)
0926 19:33:07.523873 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:07.523882 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:07.524611 (+ 729us) server_negotiation.cc:407] Connection header received
0926 19:33:07.524750 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:07.524759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:07.524918 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:07.525066 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:07.526073 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.527608 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.529074 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.529675 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.529818 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:07.532274 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:07.532286 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:07.532291 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:07.532321 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:07.535598 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:07.536476 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:07.536485 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:07.536491 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:07.536653 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:07.537381 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:07.537389 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:07.537393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:07.537916 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:07.538065 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:07.538492 (+ 427us) server_negotiation.cc:299] Negotiation successful
0926 19:33:07.538714 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 19:33:07.539194 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:07.523264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42831 (local address 127.6.215.194:57909)
0926 19:33:07.524480 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:07.524507 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:07.524634 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:07.525116 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:07.525123 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:07.525575 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:07.525895 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:07.525911 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.527784 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.527789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:07.528916 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:07.528932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:07.529968 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:07.529979 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:07.530246 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:07.530975 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:07.530993 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:07.532065 (+ 1072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:07.535736 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:07.535741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:07.535745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:07.536344 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:07.536839 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:07.536848 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:07.536854 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:07.537109 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:07.538035 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:07.538040 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:07.538245 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 19:33:07.538404 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 19:33:07.538752 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":836,"threads_started":1}
I20260926 19:33:07.545992 8344 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:42753
I20260926 19:33:07.546015 8478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:42753 every 8 connection(s)
I20260926 19:33:07.547231 8344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:07.549600 8479 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42831
I20260926 19:33:07.549839 8479 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:07.550629 8479 heartbeater.cc:511] Master 127.6.215.254:42831 requested a full tablet report, sending...
I20260926 19:33:07.552269 8133 ts_manager.cc:194] Registered new tserver with Master: 16c411e316f94a579fd5b838c68fff7b (127.6.215.194:42753)
I20260926 19:33:07.553485 8133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:43117
I20260926 19:33:07.556079 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 8344
I20260926 19:33:07.556243 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:07.556622 7007 external_mini_cluster.cc:1428] Setting key 33147540a6f2650212857cd70bee4ec2
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:07.600419 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42831
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:07.924222 8483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:07.924502 8483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:07.924656 8483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:07.932035 8483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:07.932150 8483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:07.932318 8483 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:07.932456 8483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:07.942297 8483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:42831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:07.943600 8483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:07.944926 8483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:07.957782 8489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:07.957878 8488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:07.957914 8491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:07.958935 8483 server_base.cc:1034] running on GCE node
I20260926 19:33:07.959499 8483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:07.960414 8483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:07.961624 8483 hybrid_clock.cc:648] HybridClock initialized: now 1790451187961572 us; error 29 us; skew 500 ppm
I20260926 19:33:07.961941 8483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:07 dist-test-slave-phbh krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451187, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:07.965153 8483 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:07.966749 8483 webserver.cc:460] Webserver started at http://127.6.215.195:44191/ using document root <none> and password file <none>
I20260926 19:33:07.967468 8483 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:07.967600 8483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:07.967923 8483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:07.970875 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c"
format_stamp: "Formatted at 2026-09-26 19:33:07 on dist-test-slave-phbh"
server_key: "6744d54badf99d957908c79c2b65b163"
server_key_iv: "8b42017ce7137a4d6574e654656f640e"
server_key_version: "encryptionkey@0"
I20260926 19:33:07.971710 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c"
format_stamp: "Formatted at 2026-09-26 19:33:07 on dist-test-slave-phbh"
server_key: "6744d54badf99d957908c79c2b65b163"
server_key_iv: "8b42017ce7137a4d6574e654656f640e"
server_key_version: "encryptionkey@0"
I20260926 19:33:07.977275 8483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 19:33:07.980839 8498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:07.982205 8483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:33:07.982393 8483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c"
format_stamp: "Formatted at 2026-09-26 19:33:07 on dist-test-slave-phbh"
server_key: "6744d54badf99d957908c79c2b65b163"
server_key_iv: "8b42017ce7137a4d6574e654656f640e"
server_key_version: "encryptionkey@0"
I20260926 19:33:07.982618 8483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:08.003698 8483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:08.008231 8483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:08.008525 8483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:08.011654 8483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:08.011776 8483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:08.011919 8483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:08.011997 8483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:08.021531 8340 heartbeater.cc:503] Master 127.6.215.254:42831 was elected leader, sending a full tablet report...
Sep 26 19:33:08 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451187, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:08.037766 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.012289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:57667 (local address 127.6.215.254:42831)
0926 19:33:08.013018 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.013024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.014143 (+ 1119us) server_negotiation.cc:407] Connection header received
0926 19:33:08.017228 (+ 3085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.017234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.017317 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.017439 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:08.019985 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.021060 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.023794 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.024342 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.024484 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.029987 (+ 5503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:08.029994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:08.029999 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:08.030042 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:08.033539 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.034639 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.034646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.034649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.034829 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.035399 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.035408 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.035413 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.035887 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:08.036068 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.036935 (+ 867us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.037206 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581}
I20260926 19:33:08.038599 8507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.012970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42831 (local address 127.6.215.195:57667)
0926 19:33:08.013953 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.014008 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.016852 (+ 2844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.017781 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.017797 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.018709 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:08.019751 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.019784 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.021257 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.021265 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.023569 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.023581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.024609 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.024617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.024947 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.025839 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:08.025885 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:08.029705 (+ 3820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:08.033696 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:08.033710 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:08.033733 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:08.034496 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:08.034986 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:08.034993 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:08.035003 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:08.035260 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:08.036035 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:08.036041 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:08.036439 (+ 398us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.036719 (+ 280us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.037365 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":434,"threads_started":1}
I20260926 19:33:08.062521 8483 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:46251
I20260926 19:33:08.062551 8617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:46251 every 8 connection(s)
I20260926 19:33:08.063822 8483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:08.065128 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.046795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54639 (local address 127.6.215.254:42831)
0926 19:33:08.046996 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.047003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.047895 (+ 892us) server_negotiation.cc:407] Connection header received
0926 19:33:08.048212 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.048219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.048340 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.048428 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:08.050659 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.051869 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.054300 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.054874 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.055022 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.058262 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:08.058268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:08.058272 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:08.058322 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:08.062049 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.063062 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.063067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.063069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.063194 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.063568 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.063578 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.063580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.063850 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:08.064012 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.064512 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.064735 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:33:08.066105 8618 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42831
I20260926 19:33:08.066219 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.047013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42831 (local address 127.6.215.195:54639)
0926 19:33:08.047757 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.047783 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.048027 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.048677 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.048683 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.049718 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:08.050418 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.050438 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.052078 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.052092 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.054156 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.054172 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.055155 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.055161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.055442 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.056391 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:08.056423 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:08.057995 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:08.062207 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:08.062215 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:08.062222 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:08.062927 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:08.063309 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:08.063315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:08.063319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:08.063443 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:08.063972 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:08.063977 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:08.064165 (+ 188us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.065318 (+ 1153us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.065703 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":394,"threads_started":1}
I20260926 19:33:08.066346 8618 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:08.067104 8618 heartbeater.cc:511] Master 127.6.215.254:42831 requested a full tablet report, sending...
I20260926 19:33:08.068205 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 8483
I20260926 19:33:08.068405 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:08.068956 7007 external_mini_cluster.cc:1428] Setting key 4d6eff6187d3b7bf5322edb6014f9b49
I20260926 19:33:08.070150 8133 ts_manager.cc:194] Registered new tserver with Master: 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251)
I20260926 19:33:08.070948 8133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:57667
I20260926 19:33:08.071915 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 19:33:08 dist-test-slave-phbh krb5kdc[8088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 19:33:08 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:08.116727 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.103320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.6.215.254:42831)
0926 19:33:08.103485 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.103489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.103859 (+ 370us) server_negotiation.cc:407] Connection header received
0926 19:33:08.104013 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.104017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.104091 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.104183 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:08.105704 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.106444 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.107660 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.108075 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.108165 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.111927 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:08.111932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:08.111935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:08.111966 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:08.114336 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.115016 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.115020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.115022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.115144 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.115557 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.115560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.115562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.115804 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:08.115970 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.116239 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.116398 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:33:08.121150 8132 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:08.127975 8132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:08.156356 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.140723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.6.215.194:42753)
0926 19:33:08.146175 (+ 5452us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.146188 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.146225 (+ 37us) server_negotiation.cc:407] Connection header received
0926 19:33:08.146360 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.146371 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.146702 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.146944 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:08.147806 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.149842 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.153576 (+ 3734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.155049 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.155199 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.155451 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.155874 (+ 423us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.156041 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5230,"thread_start_us":5050,"threads_started":1}
I20260926 19:33:08.156459 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.140174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:42753 (local address 127.0.0.1:48198)
0926 19:33:08.145408 (+ 5234us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.145432 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.145573 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.146995 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.147007 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.147053 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:08.147531 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.147548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.150043 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.150051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.153231 (+ 3180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.153252 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.155386 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.155392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.155607 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.155665 (+ 58us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.155817 (+ 152us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.156001 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5042,"mutex_wait_us":2,"thread_start_us":778,"threads_started":1}
I20260926 19:33:08.157022 8632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.140010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:46251 (local address 127.0.0.1:56790)
0926 19:33:08.141007 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.141067 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.141307 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.148346 (+ 7039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.148355 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.148388 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:08.148963 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.148978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.151909 (+ 2931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.151914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.154534 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.154549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.156339 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.156346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.156555 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.156586 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.156659 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.156746 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":4987,"threads_started":1}
I20260926 19:33:08.158083 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.140273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.6.215.195:46251)
0926 19:33:08.147669 (+ 7396us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.147676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.147705 (+ 29us) server_negotiation.cc:407] Connection header received
0926 19:33:08.147801 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.147811 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.148079 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.149224 (+ 1145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:08.149594 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.151761 (+ 2167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.154731 (+ 2970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.156111 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.157351 (+ 1240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.157561 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.157630 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.157784 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7256,"thread_start_us":7072,"threads_started":1}
I20260926 19:33:08.160426 8409 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd4481fde2d40d68bc285330f5aa516 (DEFAULT_TABLE table=test-table [id=38998a39f1334e0290889476c87922c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:08.160967 8548 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd4481fde2d40d68bc285330f5aa516 (DEFAULT_TABLE table=test-table [id=38998a39f1334e0290889476c87922c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:08.161840 8409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd4481fde2d40d68bc285330f5aa516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.162333 8548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd4481fde2d40d68bc285330f5aa516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.163488 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.147694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:43043 (local address 127.0.0.1:54274)
0926 19:33:08.154474 (+ 6780us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.154494 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.154614 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.155274 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.155279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.155303 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:08.155692 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.155700 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.158035 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.158040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.160149 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.160163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.162672 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.162679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.162939 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.162976 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.163048 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.163165 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6649,"thread_start_us":6504,"threads_started":1}
I20260926 19:33:08.164572 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.145788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.6.215.193:43043)
0926 19:33:08.154171 (+ 8383us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.154179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.154726 (+ 547us) server_negotiation.cc:407] Connection header received
0926 19:33:08.154824 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.154832 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.155076 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.155910 (+ 834us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:08.156097 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.157861 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.160334 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.161741 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.163843 (+ 2102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.164058 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.164121 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.164271 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8225,"thread_start_us":8090,"threads_started":1}
I20260926 19:33:08.167901 8270 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd4481fde2d40d68bc285330f5aa516 (DEFAULT_TABLE table=test-table [id=38998a39f1334e0290889476c87922c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:08.169237 8270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd4481fde2d40d68bc285330f5aa516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.173548 8637 tablet_bootstrap.cc:492] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Bootstrap starting.
I20260926 19:33:08.176121 8638 tablet_bootstrap.cc:492] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Bootstrap starting.
I20260926 19:33:08.177706 8637 tablet_bootstrap.cc:654] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.178887 8638 tablet_bootstrap.cc:654] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.178973 8637 log.cc:824] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:08.180060 8638 log.cc:824] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:08.182134 8637 tablet_bootstrap.cc:492] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: No bootstrap required, opened a new log
I20260926 19:33:08.182420 8637 ts_tablet_manager.cc:1397] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 19:33:08.183135 8638 tablet_bootstrap.cc:492] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: No bootstrap required, opened a new log
I20260926 19:33:08.183178 8641 tablet_bootstrap.cc:492] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Bootstrap starting.
I20260926 19:33:08.183396 8638 ts_tablet_manager.cc:1397] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 19:33:08.186367 8641 tablet_bootstrap.cc:654] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.187561 8641 log.cc:824] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:08.190039 8641 tablet_bootstrap.cc:492] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: No bootstrap required, opened a new log
I20260926 19:33:08.190299 8641 ts_tablet_manager.cc:1397] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 19:33:08.194147 8638 raft_consensus.cc:348] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.194147 8637 raft_consensus.cc:348] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.194576 8638 raft_consensus.cc:374] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.194578 8637 raft_consensus.cc:374] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.194680 8638 raft_consensus.cc:729] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f4cdcc48bf04cdf9c32c8fa53b20e7c, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.194696 8637 raft_consensus.cc:729] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c411e316f94a579fd5b838c68fff7b, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.195297 8638 consensus_queue.cc:260] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.195299 8637 consensus_queue.cc:260] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.196825 8479 heartbeater.cc:503] Master 127.6.215.254:42831 was elected leader, sending a full tablet report...
I20260926 19:33:08.196868 8618 heartbeater.cc:503] Master 127.6.215.254:42831 was elected leader, sending a full tablet report...
I20260926 19:33:08.197198 8638 ts_tablet_manager.cc:1428] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260926 19:33:08.198719 8637 ts_tablet_manager.cc:1428] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260926 19:33:08.198930 8641 raft_consensus.cc:348] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.199329 8641 raft_consensus.cc:374] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.199425 8641 raft_consensus.cc:729] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ec3420a674fbea8676175121989d1, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.200071 8641 consensus_queue.cc:260] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.202934 8641 ts_tablet_manager.cc:1428] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260926 19:33:08.207593 8644 raft_consensus.cc:482] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:08.207847 8644 raft_consensus.cc:504] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.209415 8644 leader_election.cc:288] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251), 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043)
I20260926 19:33:08.216550 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.209658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:49637 (local address 127.6.215.195:46251)
0926 19:33:08.209838 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.209843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.210399 (+ 556us) server_negotiation.cc:407] Connection header received
0926 19:33:08.210607 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.210613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.210689 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.210834 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:08.211464 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.212441 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.214630 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.215432 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.215609 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.215676 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.216180 (+ 504us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.216301 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 19:33:08.216605 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.209618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:46251 (local address 127.6.215.194:49637)
0926 19:33:08.210298 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.210316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.210445 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.210857 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.210862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.210891 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:08.211272 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.211333 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.212565 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.212571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.214484 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.214495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.215641 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.215651 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.216018 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.216045 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.216145 (+ 100us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.216282 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":307,"threads_started":1}
I20260926 19:33:08.216768 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.210495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:32915 (local address 127.6.215.193:43043)
0926 19:33:08.210675 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.210681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.211107 (+ 426us) server_negotiation.cc:407] Connection header received
0926 19:33:08.211216 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.211222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.211303 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.211401 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:08.212040 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.212864 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.214760 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.215592 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.215742 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.215829 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.216411 (+ 582us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.216547 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:33:08.217418 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.210385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:43043 (local address 127.6.215.194:32915)
0926 19:33:08.211012 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.211025 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.211123 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.211446 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.211450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.211472 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:08.211848 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.211859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.213006 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.213011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.214610 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.214623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.216045 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.216052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.216234 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.216265 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.216989 (+ 724us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.217123 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":4,"thread_start_us":340,"threads_started":1}
I20260926 19:33:08.217659 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd4481fde2d40d68bc285330f5aa516" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" is_pre_election: true
I20260926 19:33:08.218168 8572 raft_consensus.cc:2454] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 0.
I20260926 19:33:08.218287 8290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd4481fde2d40d68bc285330f5aa516" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ec3420a674fbea8676175121989d1" is_pre_election: true
I20260926 19:33:08.218784 8290 raft_consensus.cc:2454] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 0.
I20260926 19:33:08.218911 8362 leader_election.cc:302] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c411e316f94a579fd5b838c68fff7b, 2f4cdcc48bf04cdf9c32c8fa53b20e7c; no voters:
I20260926 19:33:08.219347 8644 raft_consensus.cc:2781] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:08.219463 8644 raft_consensus.cc:482] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:08.219612 8644 raft_consensus.cc:3037] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.221853 8644 raft_consensus.cc:504] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.222398 8644 leader_election.cc:288] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 election: Requested vote from peers 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251), 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043)
I20260926 19:33:08.222851 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd4481fde2d40d68bc285330f5aa516" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c"
I20260926 19:33:08.222958 8290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd4481fde2d40d68bc285330f5aa516" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ec3420a674fbea8676175121989d1"
I20260926 19:33:08.223055 8572 raft_consensus.cc:3037] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.223181 8290 raft_consensus.cc:3037] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.225337 8572 raft_consensus.cc:2454] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 1.
I20260926 19:33:08.225399 8290 raft_consensus.cc:2454] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 1.
I20260926 19:33:08.225732 8362 leader_election.cc:302] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c411e316f94a579fd5b838c68fff7b, 9a4ec3420a674fbea8676175121989d1; no voters:
I20260926 19:33:08.226001 8644 raft_consensus.cc:2781] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:08.226456 8644 raft_consensus.cc:686] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 1 LEADER]: Becoming Leader. State: Replica: 16c411e316f94a579fd5b838c68fff7b, State: Running, Role: LEADER
I20260926 19:33:08.226895 8644 consensus_queue.cc:237] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } }
I20260926 19:33:08.231712 8132 catalog_manager.cc:5495] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 16c411e316f94a579fd5b838c68fff7b (127.6.215.194). New cstate: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } health_report { overall_health: UNKNOWN } } }
W20260926 19:33:08.264918 8341 tablet.cc:2369] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:08.270085 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.264909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.6.215.194:42753)
0926 19:33:08.265137 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.265144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.265280 (+ 136us) server_negotiation.cc:407] Connection header received
0926 19:33:08.265620 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.265624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.265697 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.265839 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:08.266159 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.266853 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.268065 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.268464 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.268554 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.269035 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:08.269522 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:08.269621 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.269715 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.269795 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:33:08.277860 8572 raft_consensus.cc:1261] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Refusing update from remote peer 16c411e316f94a579fd5b838c68fff7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:08.277917 8290 raft_consensus.cc:1261] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Refusing update from remote peer 16c411e316f94a579fd5b838c68fff7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:08.278813 8644 consensus_queue.cc:1035] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:08.279138 8648 consensus_queue.cc:1035] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:08.309512 8132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:08.309818 8132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:08.315366 8132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc: Sending DeleteTablet for 3 replicas of tablet 5fd4481fde2d40d68bc285330f5aa516
I20260926 19:33:08.317047 8548 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd4481fde2d40d68bc285330f5aa516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:08 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:56790
I20260926 19:33:08.318001 8661 tablet_replica.cc:330] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: stopping tablet replica
I20260926 19:33:08.318517 8270 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd4481fde2d40d68bc285330f5aa516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:08 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:54274
I20260926 19:33:08.318602 8661 raft_consensus.cc:2229] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:08.319202 8661 raft_consensus.cc:2258] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:08.319602 8662 tablet_replica.cc:330] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: stopping tablet replica
I20260926 19:33:08.320003 8662 raft_consensus.cc:2229] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:08.320712 8662 raft_consensus.cc:2258] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:08.320852 8409 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd4481fde2d40d68bc285330f5aa516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:08 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:48198
I20260926 19:33:08.320809 8132 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:08.321573 8132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:08.323374 8662 ts_tablet_manager.cc:1905] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:08.329277 8663 tablet_replica.cc:330] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: stopping tablet replica
I20260926 19:33:08.329767 8663 raft_consensus.cc:2229] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:08.330359 8663 raft_consensus.cc:2258] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:08.330969 8661 ts_tablet_manager.cc:1905] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:08.332227 8663 ts_tablet_manager.cc:1905] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:08.338480 8270 tablet_service.cc:1467] Processing CreateTablet for tablet 42d0c50347e941a79b15965ca5f1ebda (DEFAULT_TABLE table=test-table [id=a3f7947013794a61844f8e813624740a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:08.338858 8661 ts_tablet_manager.cc:1918] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:08.338882 8409 tablet_service.cc:1467] Processing CreateTablet for tablet 42d0c50347e941a79b15965ca5f1ebda (DEFAULT_TABLE table=test-table [id=a3f7947013794a61844f8e813624740a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:08.339020 8661 log.cc:1196] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/5fd4481fde2d40d68bc285330f5aa516
I20260926 19:33:08.339224 8663 ts_tablet_manager.cc:1918] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:08.339327 8409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42d0c50347e941a79b15965ca5f1ebda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.339401 8663 log.cc:1196] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/5fd4481fde2d40d68bc285330f5aa516
I20260926 19:33:08.339524 8661 ts_tablet_manager.cc:1939] T 5fd4481fde2d40d68bc285330f5aa516 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Deleting consensus metadata
I20260926 19:33:08.339958 8663 ts_tablet_manager.cc:1939] T 5fd4481fde2d40d68bc285330f5aa516 P 16c411e316f94a579fd5b838c68fff7b: Deleting consensus metadata
I20260926 19:33:08.341603 8548 tablet_service.cc:1467] Processing CreateTablet for tablet 42d0c50347e941a79b15965ca5f1ebda (DEFAULT_TABLE table=test-table [id=a3f7947013794a61844f8e813624740a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:08.342022 8548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42d0c50347e941a79b15965ca5f1ebda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.342656 8117 catalog_manager.cc:4841] TS 16c411e316f94a579fd5b838c68fff7b (127.6.215.194:42753): tablet 5fd4481fde2d40d68bc285330f5aa516 (table test-table [id=38998a39f1334e0290889476c87922c9]) successfully deleted
I20260926 19:33:08.346520 8637 tablet_bootstrap.cc:492] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: Bootstrap starting.
I20260926 19:33:08.347772 8638 tablet_bootstrap.cc:492] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Bootstrap starting.
I20260926 19:33:08.348042 8637 tablet_bootstrap.cc:654] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.348999 8638 tablet_bootstrap.cc:654] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.349279 8270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42d0c50347e941a79b15965ca5f1ebda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.349984 8119 catalog_manager.cc:4841] TS 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251): tablet 5fd4481fde2d40d68bc285330f5aa516 (table test-table [id=38998a39f1334e0290889476c87922c9]) successfully deleted
I20260926 19:33:08.356642 8641 tablet_bootstrap.cc:492] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: Bootstrap starting.
I20260926 19:33:08.358004 8662 ts_tablet_manager.cc:1918] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:08.358115 8637 tablet_bootstrap.cc:492] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: No bootstrap required, opened a new log
I20260926 19:33:08.358222 8662 log.cc:1196] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/5fd4481fde2d40d68bc285330f5aa516
I20260926 19:33:08.358305 8637 ts_tablet_manager.cc:1397] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20260926 19:33:08.358512 8641 tablet_bootstrap.cc:654] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.358876 8662 ts_tablet_manager.cc:1939] T 5fd4481fde2d40d68bc285330f5aa516 P 9a4ec3420a674fbea8676175121989d1: Deleting consensus metadata
I20260926 19:33:08.359261 8637 raft_consensus.cc:348] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.359494 8637 raft_consensus.cc:374] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.359591 8637 raft_consensus.cc:729] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c411e316f94a579fd5b838c68fff7b, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.359903 8637 consensus_queue.cc:260] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.360460 8637 ts_tablet_manager.cc:1428] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260926 19:33:08.360880 8119 catalog_manager.cc:4841] TS 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043): tablet 5fd4481fde2d40d68bc285330f5aa516 (table test-table [id=38998a39f1334e0290889476c87922c9]) successfully deleted
I20260926 19:33:08.364182 8638 tablet_bootstrap.cc:492] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: No bootstrap required, opened a new log
I20260926 19:33:08.364352 8638 ts_tablet_manager.cc:1397] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.007s
I20260926 19:33:08.365242 8638 raft_consensus.cc:348] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.365475 8638 raft_consensus.cc:374] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.365576 8638 raft_consensus.cc:729] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f4cdcc48bf04cdf9c32c8fa53b20e7c, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.365860 8638 consensus_queue.cc:260] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.366451 8638 ts_tablet_manager.cc:1428] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 19:33:08.370628 8641 tablet_bootstrap.cc:492] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: No bootstrap required, opened a new log
I20260926 19:33:08.370800 8641 ts_tablet_manager.cc:1397] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20260926 19:33:08.371678 8641 raft_consensus.cc:348] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.371920 8641 raft_consensus.cc:374] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.372010 8641 raft_consensus.cc:729] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ec3420a674fbea8676175121989d1, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.372351 8641 consensus_queue.cc:260] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.374481 8641 ts_tablet_manager.cc:1428] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260926 19:33:08.381217 8656 raft_consensus.cc:482] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:08.381484 8656 raft_consensus.cc:504] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.382432 8656 leader_election.cc:288] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043), 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251)
I20260926 19:33:08.385367 8290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42d0c50347e941a79b15965ca5f1ebda" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ec3420a674fbea8676175121989d1" is_pre_election: true
I20260926 19:33:08.385468 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42d0c50347e941a79b15965ca5f1ebda" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" is_pre_election: true
I20260926 19:33:08.385637 8290 raft_consensus.cc:2454] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 0.
I20260926 19:33:08.385704 8572 raft_consensus.cc:2454] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 0.
I20260926 19:33:08.386112 8362 leader_election.cc:302] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c411e316f94a579fd5b838c68fff7b, 2f4cdcc48bf04cdf9c32c8fa53b20e7c; no voters:
I20260926 19:33:08.386407 8656 raft_consensus.cc:2781] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:08.386547 8656 raft_consensus.cc:482] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:08.386699 8656 raft_consensus.cc:3037] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.392170 8656 raft_consensus.cc:504] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.393352 8290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42d0c50347e941a79b15965ca5f1ebda" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ec3420a674fbea8676175121989d1"
I20260926 19:33:08.393415 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42d0c50347e941a79b15965ca5f1ebda" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c"
I20260926 19:33:08.393563 8290 raft_consensus.cc:3037] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.393610 8572 raft_consensus.cc:3037] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.396567 8572 raft_consensus.cc:2454] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 1.
I20260926 19:33:08.396659 8290 raft_consensus.cc:2454] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 1.
I20260926 19:33:08.392961 8656 leader_election.cc:288] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 election: Requested vote from peers 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043), 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251)
I20260926 19:33:08.397068 8362 leader_election.cc:302] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c411e316f94a579fd5b838c68fff7b, 9a4ec3420a674fbea8676175121989d1; no voters:
I20260926 19:33:08.397603 8656 raft_consensus.cc:2781] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:08.398223 8656 raft_consensus.cc:686] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 1 LEADER]: Becoming Leader. State: Replica: 16c411e316f94a579fd5b838c68fff7b, State: Running, Role: LEADER
I20260926 19:33:08.398612 8656 consensus_queue.cc:237] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.401861 8132 catalog_manager.cc:5495] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 16c411e316f94a579fd5b838c68fff7b (127.6.215.194). New cstate: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } health_report { overall_health: UNKNOWN } } }
Sep 26 19:33:08 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451186, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:08.463150 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.447469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42831 (local address 127.0.0.1:54288)
0926 19:33:08.448152 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.448168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.448294 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.448659 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.448662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.450036 (+ 1374us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:08.450300 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.450308 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.451787 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.451791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.452730 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.452740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.453801 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.453806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.453924 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.454599 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:08.454616 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:08.456967 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:08.460560 (+ 3593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:08.460570 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:08.460576 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:08.461097 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:08.461512 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:08.461515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:08.461517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:08.461656 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:08.462309 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:08.462315 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:08.462461 (+ 146us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.462593 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.462809 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":379,"threads_started":1}
I20260926 19:33:08.463495 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.447451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.6.215.254:42831)
0926 19:33:08.447666 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.447672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.448260 (+ 588us) server_negotiation.cc:407] Connection header received
0926 19:33:08.448412 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.448418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.448506 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.448624 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:08.450459 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.451634 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.452939 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.453624 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.453761 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.457147 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:08.457154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:08.457159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:08.457194 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:08.460388 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.461227 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.461233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.461237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.461376 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:08.461776 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:08.461781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:08.461785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:08.462138 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:08.462281 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.462933 (+ 652us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.463153 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:33:08.468935 8132 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:54288:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 19:33:08.470126 8132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:08.477180 8270 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6383bd979441a5ae6a400bb7f63e88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=367cc183ff0f4599ad12411ac363c0e1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:33:08.477376 8409 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6383bd979441a5ae6a400bb7f63e88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=367cc183ff0f4599ad12411ac363c0e1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:33:08.477497 8548 tablet_service.cc:1467] Processing CreateTablet for tablet 2a6383bd979441a5ae6a400bb7f63e88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=367cc183ff0f4599ad12411ac363c0e1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:33:08.477757 8270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6383bd979441a5ae6a400bb7f63e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.477937 8409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6383bd979441a5ae6a400bb7f63e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.477958 8548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a6383bd979441a5ae6a400bb7f63e88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:08.486304 8638 tablet_bootstrap.cc:492] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Bootstrap starting.
I20260926 19:33:08.486708 8641 tablet_bootstrap.cc:492] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: Bootstrap starting.
I20260926 19:33:08.486829 8637 tablet_bootstrap.cc:492] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: Bootstrap starting.
I20260926 19:33:08.488173 8638 tablet_bootstrap.cc:654] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.488346 8641 tablet_bootstrap.cc:654] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.489230 8637 tablet_bootstrap.cc:654] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:08.495229 8641 tablet_bootstrap.cc:492] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: No bootstrap required, opened a new log
I20260926 19:33:08.495402 8641 ts_tablet_manager.cc:1397] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260926 19:33:08.497123 8641 raft_consensus.cc:348] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.497380 8641 raft_consensus.cc:374] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.497468 8641 raft_consensus.cc:729] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4ec3420a674fbea8676175121989d1, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.497742 8641 consensus_queue.cc:260] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.498220 8645 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.498538 8645 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:08.501827 8641 ts_tablet_manager.cc:1428] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260926 19:33:08.502506 8638 tablet_bootstrap.cc:492] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: No bootstrap required, opened a new log
I20260926 19:33:08.502691 8638 ts_tablet_manager.cc:1397] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260926 19:33:08.503098 8637 tablet_bootstrap.cc:492] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: No bootstrap required, opened a new log
I20260926 19:33:08.503351 8637 ts_tablet_manager.cc:1397] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260926 19:33:08.504007 8638 raft_consensus.cc:348] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.504280 8638 raft_consensus.cc:374] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.504391 8638 raft_consensus.cc:729] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f4cdcc48bf04cdf9c32c8fa53b20e7c, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.504199 8637 raft_consensus.cc:348] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.504695 8637 raft_consensus.cc:374] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:08.504649 8638 consensus_queue.cc:260] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.504880 8637 raft_consensus.cc:729] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c411e316f94a579fd5b838c68fff7b, State: Initialized, Role: FOLLOWER
I20260926 19:33:08.505378 8638 ts_tablet_manager.cc:1428] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:08.505201 8643 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.505599 8643 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:08.505288 8637 consensus_queue.cc:260] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.505932 8656 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.506227 8656 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:08.506726 8637 ts_tablet_manager.cc:1428] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260926 19:33:08.550130 8480 tablet.cc:2369] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:33:08.567575 8619 tablet.cc:2369] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:08.805271 8656 consensus_queue.cc:1035] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:08.809973 8679 consensus_queue.cc:1035] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:08.848125 8644 raft_consensus.cc:482] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:08.848313 8644 raft_consensus.cc:504] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.848948 8644 leader_election.cc:288] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043), 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251)
I20260926 19:33:08.849474 8290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6383bd979441a5ae6a400bb7f63e88" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ec3420a674fbea8676175121989d1" is_pre_election: true
I20260926 19:33:08.849498 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6383bd979441a5ae6a400bb7f63e88" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" is_pre_election: true
I20260926 19:33:08.849742 8572 raft_consensus.cc:2454] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 0.
I20260926 19:33:08.849742 8290 raft_consensus.cc:2454] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 0.
I20260926 19:33:08.850248 8362 leader_election.cc:302] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c411e316f94a579fd5b838c68fff7b, 9a4ec3420a674fbea8676175121989d1; no voters:
I20260926 19:33:08.850538 8644 raft_consensus.cc:2781] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:08.850679 8644 raft_consensus.cc:482] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:08.850773 8644 raft_consensus.cc:3037] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.852722 8644 raft_consensus.cc:504] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.853227 8644 leader_election.cc:288] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 election: Requested vote from peers 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043), 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251)
I20260926 19:33:08.853778 8290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6383bd979441a5ae6a400bb7f63e88" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4ec3420a674fbea8676175121989d1"
I20260926 19:33:08.853801 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a6383bd979441a5ae6a400bb7f63e88" candidate_uuid: "16c411e316f94a579fd5b838c68fff7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c"
I20260926 19:33:08.854043 8290 raft_consensus.cc:3037] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.854048 8572 raft_consensus.cc:3037] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:08.856186 8572 raft_consensus.cc:2454] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 1.
I20260926 19:33:08.856184 8290 raft_consensus.cc:2454] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16c411e316f94a579fd5b838c68fff7b in term 1.
I20260926 19:33:08.856565 8362 leader_election.cc:302] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c411e316f94a579fd5b838c68fff7b, 9a4ec3420a674fbea8676175121989d1; no voters:
I20260926 19:33:08.856978 8644 raft_consensus.cc:2781] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:08.857167 8644 raft_consensus.cc:686] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [term 1 LEADER]: Becoming Leader. State: Replica: 16c411e316f94a579fd5b838c68fff7b, State: Running, Role: LEADER
I20260926 19:33:08.857438 8644 consensus_queue.cc:237] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } }
I20260926 19:33:08.859021 8679 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: TxnStatusTablet state changed. Reason: New leader 16c411e316f94a579fd5b838c68fff7b. Latest consensus state: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.859306 8679 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:33:08.859879 8684 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 19:33:08.860121 8684 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 19:33:08.861119 8132 catalog_manager.cc:5495] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 16c411e316f94a579fd5b838c68fff7b (127.6.215.194). New cstate: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } health_report { overall_health: UNKNOWN } } }
I20260926 19:33:08.955687 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.949995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:42753 (local address 127.0.0.1:48212)
0926 19:33:08.950257 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.950308 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.950474 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.951360 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.951366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.951379 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:33:08.951584 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.951593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.952732 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.952739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.953761 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.953780 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.954524 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.954530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.954696 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.954746 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:33:08.955136 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:33:08.955156 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.955235 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.955386 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 19:33:08.955752 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.950131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.6.215.194:42753)
0926 19:33:08.951063 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.951071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.951085 (+ 14us) server_negotiation.cc:407] Connection header received
0926 19:33:08.951138 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.951142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.951199 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.951316 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:08.951743 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.952599 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.953906 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.954374 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.954502 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.954928 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:08.955026 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:08.955124 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.955291 (+ 167us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.955365 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":643,"threads_started":1}
I20260926 19:33:08.959996 8572 raft_consensus.cc:1261] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Refusing update from remote peer 16c411e316f94a579fd5b838c68fff7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:08.960129 8290 raft_consensus.cc:1261] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Refusing update from remote peer 16c411e316f94a579fd5b838c68fff7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:08.960691 8644 consensus_queue.cc:1035] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:08.961045 8679 consensus_queue.cc:1035] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:08.964744 8643 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: TxnStatusTablet state changed. Reason: New leader 16c411e316f94a579fd5b838c68fff7b. Latest consensus state: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.965044 8643 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:08.966761 8682 mvcc.cc:205] Tried to move back new op lower bound from 7333688069973975040 to 7333688069562454016. Current Snapshot: MvccSnapshot[applied={T|T < 7333688069973975040}]
I20260926 19:33:08.968315 8678 mvcc.cc:205] Tried to move back new op lower bound from 7333688069973975040 to 7333688069562454016. Current Snapshot: MvccSnapshot[applied={T|T < 7333688069973975040 or (T in {7333688069973975040})}]
I20260926 19:33:08.968870 8656 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.969120 8656 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:33:08.970489 8645 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: TxnStatusTablet state changed. Reason: New leader 16c411e316f94a579fd5b838c68fff7b. Latest consensus state: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.970677 8645 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:08.974329 8643 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.974551 8643 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:08.977594 8656 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.978335 8656 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 16c411e316f94a579fd5b838c68fff7b: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:33:08.979012 8680 mvcc.cc:205] Tried to move back new op lower bound from 7333688069973975040 to 7333688069562454016. Current Snapshot: MvccSnapshot[applied={T|T < 7333688069973975040}]
I20260926 19:33:08.979535 8645 tablet_replica.cc:439] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "16c411e316f94a579fd5b838c68fff7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a4ec3420a674fbea8676175121989d1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43043 } } peers { permanent_uuid: "16c411e316f94a579fd5b838c68fff7b" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 42753 } } peers { permanent_uuid: "2f4cdcc48bf04cdf9c32c8fa53b20e7c" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46251 } } }
I20260926 19:33:08.979781 8645 tablet_replica.cc:442] T 2a6383bd979441a5ae6a400bb7f63e88 P 9a4ec3420a674fbea8676175121989d1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:08.992514 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.986278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:51787 (local address 127.6.215.194:42753)
0926 19:33:08.986446 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:08.986450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:08.987207 (+ 757us) server_negotiation.cc:407] Connection header received
0926 19:33:08.987409 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:08.987414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:08.987482 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:08.987595 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:08.988143 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.988948 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.990840 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.991244 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.991383 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.991800 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:08.991897 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:08.991989 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:08.992126 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 19:33:08.992187 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 19:33:08.992658 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:08.986361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:42753 (local address 127.6.215.194:51787)
0926 19:33:08.987119 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:08.987134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:08.987286 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:08.987624 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:08.987628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:08.987640 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:33:08.987987 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.987996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.989069 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.989081 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.990693 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:08.990708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:08.991426 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:08.991434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:08.991586 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:08.991634 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:33:08.991999 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:33:08.992010 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 19:33:08.992100 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 19:33:08.992226 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":360,"threads_started":1}
I20260926 19:33:09.061664 8131 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:09.061919 8131 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:09.064846 8131 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e31334c16a74ea39d6268ca9697d6cc: Sending DeleteTablet for 3 replicas of tablet 42d0c50347e941a79b15965ca5f1ebda
I20260926 19:33:09.065778 8548 tablet_service.cc:1514] Processing DeleteTablet for tablet 42d0c50347e941a79b15965ca5f1ebda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:09 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:56790
I20260926 19:33:09.067632 8270 tablet_service.cc:1514] Processing DeleteTablet for tablet 42d0c50347e941a79b15965ca5f1ebda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:09 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:54274
I20260926 19:33:09.074700 8408 tablet_service.cc:1514] Processing DeleteTablet for tablet 42d0c50347e941a79b15965ca5f1ebda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:09 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:48198
I20260926 19:33:09.080400 8699 tablet_replica.cc:330] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: stopping tablet replica
I20260926 19:33:09.080850 8699 raft_consensus.cc:2229] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:09.081264 8699 raft_consensus.cc:2258] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:09.082824 8699 ts_tablet_manager.cc:1905] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:09.088299 8697 tablet_replica.cc:330] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: stopping tablet replica
I20260926 19:33:09.093014 8699 ts_tablet_manager.cc:1918] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:09.093431 8697 raft_consensus.cc:2229] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:09.093793 8697 raft_consensus.cc:2258] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:09.095084 8697 ts_tablet_manager.cc:1905] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:09.095536 8700 tablet_replica.cc:330] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: stopping tablet replica
I20260926 19:33:09.095737 8700 raft_consensus.cc:2229] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:09.096168 8700 raft_consensus.cc:2258] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:09.097288 8700 ts_tablet_manager.cc:1905] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:09.099833 8699 log.cc:1196] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/42d0c50347e941a79b15965ca5f1ebda
I20260926 19:33:09.100351 8699 ts_tablet_manager.cc:1939] T 42d0c50347e941a79b15965ca5f1ebda P 9a4ec3420a674fbea8676175121989d1: Deleting consensus metadata
I20260926 19:33:09.101678 8119 catalog_manager.cc:4841] TS 9a4ec3420a674fbea8676175121989d1 (127.6.215.193:43043): tablet 42d0c50347e941a79b15965ca5f1ebda (table test-table [id=a3f7947013794a61844f8e813624740a]) successfully deleted
I20260926 19:33:09.105785 8697 ts_tablet_manager.cc:1918] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:09.106021 8697 log.cc:1196] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/42d0c50347e941a79b15965ca5f1ebda
Sep 26 19:33:09 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:09.106563 8697 ts_tablet_manager.cc:1939] T 42d0c50347e941a79b15965ca5f1ebda P 2f4cdcc48bf04cdf9c32c8fa53b20e7c: Deleting consensus metadata
I20260926 19:33:09.107815 8119 catalog_manager.cc:4841] TS 2f4cdcc48bf04cdf9c32c8fa53b20e7c (127.6.215.195:46251): tablet 42d0c50347e941a79b15965ca5f1ebda (table test-table [id=a3f7947013794a61844f8e813624740a]) successfully deleted
I20260926 19:33:09.112670 8700 ts_tablet_manager.cc:1918] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:09.112902 8700 log.cc:1196] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/42d0c50347e941a79b15965ca5f1ebda
I20260926 19:33:09.113492 8700 ts_tablet_manager.cc:1939] T 42d0c50347e941a79b15965ca5f1ebda P 16c411e316f94a579fd5b838c68fff7b: Deleting consensus metadata
I20260926 19:33:09.114751 8117 catalog_manager.cc:4841] TS 16c411e316f94a579fd5b838c68fff7b (127.6.215.194:42753): tablet 42d0c50347e941a79b15965ca5f1ebda (table test-table [id=a3f7947013794a61844f8e813624740a]) successfully deleted
I20260926 19:33:09.115592 8701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:09.077140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.6.215.193:43043)
0926 19:33:09.092349 (+ 15209us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:09.092356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:09.094810 (+ 2454us) server_negotiation.cc:407] Connection header received
0926 19:33:09.094988 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:09.094993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:09.095073 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:09.095196 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:09.097257 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.098827 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.100729 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.101379 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.101484 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:09.107727 (+ 6243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:09.107746 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:09.107765 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:09.107811 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:09.112121 (+ 4310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.113141 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.113155 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.113160 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.113323 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.113811 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.113816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.113819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.114311 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:09.114584 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:09.115147 (+ 563us) server_negotiation.cc:299] Negotiation successful
0926 19:33:09.115339 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15004,"thread_start_us":15180,"threads_started":1}
W20260926 19:33:09.117017 8335 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54286
I20260926 19:33:09.139277 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:09.120945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54294 (local address 127.6.215.254:42831)
0926 19:33:09.124662 (+ 3717us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:09.124669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:09.124877 (+ 208us) server_negotiation.cc:407] Connection header received
0926 19:33:09.125043 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:09.125048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:09.125131 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:09.125256 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:09.126843 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.128049 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.129461 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.130111 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.130245 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:09.132507 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:09.132514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:09.132518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:09.132563 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:09.136530 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.137221 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.137227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.137231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.137393 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.137731 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.137739 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.137744 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.138102 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:09.138374 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:09.138611 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 19:33:09.138838 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3558,"thread_start_us":3397,"threads_started":1}
W20260926 19:33:09.139873 8131 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54294
I20260926 19:33:09.141641 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8204
I20260926 19:33:09.156680 8701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:09.142732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54290 (local address 127.6.215.193:43043)
0926 19:33:09.142952 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:09.142958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:09.143357 (+ 399us) server_negotiation.cc:407] Connection header received
0926 19:33:09.143573 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:09.143581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:09.143668 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:09.143778 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:09.144835 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.145889 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.147015 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.147605 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.147789 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:09.150735 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:09.150741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:09.150746 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:09.150784 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:09.154112 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.154783 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.154790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.154796 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.154960 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.155306 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.155311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.155314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.155647 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:09.155899 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:09.156135 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 19:33:09.156350 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 19:33:09.157347 8335 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54290
W20260926 19:33:09.157931 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8204: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:09.158099 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 8204
I20260926 19:33:09.176146 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8344
Sep 26 19:33:09 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:33:09.193958 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:09.177004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.6.215.194:42753)
0926 19:33:09.177236 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:09.177243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:09.177267 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:33:09.177530 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:09.177537 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:09.177645 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:09.177771 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:09.179004 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.180198 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.181509 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.182200 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.182337 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:09.186553 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:09.186571 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:09.186593 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:09.186642 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:09.190943 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.191648 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.191664 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.191668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.191845 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.192210 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.192216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.192220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.192702 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:09.193066 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:09.193256 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 19:33:09.193483 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 19:33:09.194761 8474 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48218
W20260926 19:33:09.195546 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8344: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:09.195724 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 8344
I20260926 19:33:09.219483 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8483
Sep 26 19:33:09 dist-test-slave-phbh krb5kdc[8088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:09.234499 8707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:09.220273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56804 (local address 127.6.215.195:46251)
0926 19:33:09.220873 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:09.220877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:09.220895 (+ 18us) server_negotiation.cc:407] Connection header received
0926 19:33:09.220948 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:09.220952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:09.221014 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:09.221101 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:09.222313 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.223042 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.224275 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:09.224711 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:09.224894 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:09.229010 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:09.229027 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:09.229042 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:09.229084 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:09.231991 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.232655 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.232663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.232665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.232867 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:09.233197 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:09.233200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:09.233202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:09.233588 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:09.233794 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:09.234049 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 19:33:09.234213 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":260,"threads_started":1}
W20260926 19:33:09.235203 8613 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56804
W20260926 19:33:09.235853 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8483: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:09.236019 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 8483
I20260926 19:33:09.252400 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8101
I20260926 19:33:09.253509 8173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:09.369921 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 8101
2026-09-26T19:33:09Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4708 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:09 dist-test-slave-phbh krb5kdc[8711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:09 dist-test-slave-phbh krb5kdc[8711](info): set up 2 sockets
Sep 26 19:33:09 dist-test-slave-phbh krb5kdc[8711](info): commencing operation
krb5kdc: starting...
W20260926 19:33:10.466135 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:10 dist-test-slave-phbh krb5kdc[8711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:10.660234 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:36299
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:36299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:10.977682 8724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:10.978045 8724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:10.978228 8724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:10.985535 8724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:10.985690 8724 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 19:33:10.985792 8724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:10.985906 8724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:10.985999 8724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:10.986086 8724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:10.995638 8724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:36299
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:36299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:10.996963 8724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:10.998309 8724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:11.010087 8729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:11.010085 8732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:11.010113 8730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:11.014477 8724 server_base.cc:1034] running on GCE node
I20260926 19:33:11.015343 8724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:11.016861 8724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:11.018100 8724 hybrid_clock.cc:648] HybridClock initialized: now 1790451191018048 us; error 35 us; skew 500 ppm
I20260926 19:33:11.018484 8724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:11 dist-test-slave-phbh krb5kdc[8711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451191, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:11.021651 8724 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:11.023188 8724 webserver.cc:460] Webserver started at http://127.6.215.254:45517/ using document root <none> and password file <none>
I20260926 19:33:11.023939 8724 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:11.024070 8724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:11.024391 8724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:11.027351 8724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "e727b0c8f383404f9c08b2e202c3241f"
format_stamp: "Formatted at 2026-09-26 19:33:11 on dist-test-slave-phbh"
server_key: "67b903b7e2bb7047080a2e6eeadd4d08"
server_key_iv: "91b0626e0615969c35921e4bd6200bfc"
server_key_version: "encryptionkey@0"
I20260926 19:33:11.028086 8724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "e727b0c8f383404f9c08b2e202c3241f"
format_stamp: "Formatted at 2026-09-26 19:33:11 on dist-test-slave-phbh"
server_key: "67b903b7e2bb7047080a2e6eeadd4d08"
server_key_iv: "91b0626e0615969c35921e4bd6200bfc"
server_key_version: "encryptionkey@0"
I20260926 19:33:11.033167 8724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 19:33:11.036468 8739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:11.037883 8724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:11.038054 8724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "e727b0c8f383404f9c08b2e202c3241f"
format_stamp: "Formatted at 2026-09-26 19:33:11 on dist-test-slave-phbh"
server_key: "67b903b7e2bb7047080a2e6eeadd4d08"
server_key_iv: "91b0626e0615969c35921e4bd6200bfc"
server_key_version: "encryptionkey@0"
I20260926 19:33:11.038308 8724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:11.059242 8724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:11.063598 8724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:11.063902 8724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:11.085265 8724 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:36299
I20260926 19:33:11.085332 8800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:36299 every 8 connection(s)
I20260926 19:33:11.086649 8724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:11.088007 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 8724
I20260926 19:33:11.088193 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:11.089047 7007 external_mini_cluster.cc:1428] Setting key 4d93299dc8915a6d22200444c0f76722
I20260926 19:33:11.091418 8801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 19:33:11 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:11.104506 8801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Bootstrap starting.
I20260926 19:33:11.107661 8801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:11.108672 8801 log.cc:824] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:11.111847 8801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: No bootstrap required, opened a new log
I20260926 19:33:11.112336 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:11.091099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.6.215.254:36299)
0926 19:33:11.091951 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:11.091963 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:11.092002 (+ 39us) server_negotiation.cc:407] Connection header received
0926 19:33:11.093640 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:11.093666 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:11.094045 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:11.094373 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:11.096093 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.097535 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.099264 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.100086 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.100230 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:11.104890 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:11.104899 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:11.104913 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:11.104948 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:11.107972 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:11.108687 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:11.108699 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:11.108704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:11.108943 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:11.109389 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:11.109392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:11.109394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:11.109983 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:11.110246 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:11.110627 (+ 381us) server_negotiation.cc:299] Negotiation successful
0926 19:33:11.111041 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":384,"threads_started":1}
I20260926 19:33:11.119083 8801 raft_consensus.cc:348] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e727b0c8f383404f9c08b2e202c3241f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36299 } }
I20260926 19:33:11.119419 8801 raft_consensus.cc:374] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:11.119530 8801 raft_consensus.cc:729] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e727b0c8f383404f9c08b2e202c3241f, State: Initialized, Role: FOLLOWER
I20260926 19:33:11.120056 8801 consensus_queue.cc:260] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e727b0c8f383404f9c08b2e202c3241f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36299 } }
I20260926 19:33:11.120316 8801 raft_consensus.cc:388] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:11.120455 8801 raft_consensus.cc:482] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:11.120618 8801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:11.122628 8801 raft_consensus.cc:504] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e727b0c8f383404f9c08b2e202c3241f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36299 } }
I20260926 19:33:11.123023 8801 leader_election.cc:302] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e727b0c8f383404f9c08b2e202c3241f; no voters:
I20260926 19:33:11.123503 8801 leader_election.cc:288] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:11.123649 8806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:11.124387 8806 raft_consensus.cc:686] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [term 1 LEADER]: Becoming Leader. State: Replica: e727b0c8f383404f9c08b2e202c3241f, State: Running, Role: LEADER
I20260926 19:33:11.125111 8806 consensus_queue.cc:237] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e727b0c8f383404f9c08b2e202c3241f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36299 } }
I20260926 19:33:11.125644 8801 sys_catalog.cc:564] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:11.127697 8807 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e727b0c8f383404f9c08b2e202c3241f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e727b0c8f383404f9c08b2e202c3241f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36299 } } }
I20260926 19:33:11.128141 8807 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:11.128635 8808 sys_catalog.cc:455] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e727b0c8f383404f9c08b2e202c3241f. Latest consensus state: current_term: 1 leader_uuid: "e727b0c8f383404f9c08b2e202c3241f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e727b0c8f383404f9c08b2e202c3241f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36299 } } }
I20260926 19:33:11.129069 8808 sys_catalog.cc:458] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:11.130291 8814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:11.135388 8814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:11.144112 8814 catalog_manager.cc:1347] Generated new cluster ID: 6659f307f9254575a2cfc1c5f53f2ef3
I20260926 19:33:11.144232 8814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:11.157658 8814 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:11.158897 8814 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:11.166956 8814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Generated new TSK 0
I20260926 19:33:11.167814 8814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:11.230167 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:36299
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:11.570190 8827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:11.570519 8827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:11.570633 8827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:11.578994 8827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:11.579124 8827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:11.579216 8827 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 19:33:11.579282 8827 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:11.579357 8827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:11.590596 8827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:11.591934 8827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:11.593389 8827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:11.607456 8833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:11.607577 8832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:11.607702 8835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:11.608465 8827 server_base.cc:1034] running on GCE node
I20260926 19:33:11.609200 8827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:11.610129 8827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:11.611349 8827 hybrid_clock.cc:648] HybridClock initialized: now 1790451191611296 us; error 35 us; skew 500 ppm
I20260926 19:33:11.611666 8827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:11 dist-test-slave-phbh krb5kdc[8711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451191, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:11.614981 8827 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:11.616410 8827 webserver.cc:460] Webserver started at http://127.6.215.193:37429/ using document root <none> and password file <none>
I20260926 19:33:11.617156 8827 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:11.617264 8827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:11.617532 8827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:11.620585 8827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "b79eed4702d648bfa8dee79ecee570e1"
format_stamp: "Formatted at 2026-09-26 19:33:11 on dist-test-slave-phbh"
server_key: "398bfb4eaa0ae8ad459a58b22eb78599"
server_key_iv: "c3ee675c4ce66060e8e75b6c22e71511"
server_key_version: "encryptionkey@0"
I20260926 19:33:11.621511 8827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "b79eed4702d648bfa8dee79ecee570e1"
format_stamp: "Formatted at 2026-09-26 19:33:11 on dist-test-slave-phbh"
server_key: "398bfb4eaa0ae8ad459a58b22eb78599"
server_key_iv: "c3ee675c4ce66060e8e75b6c22e71511"
server_key_version: "encryptionkey@0"
I20260926 19:33:11.627513 8827 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 19:33:11.631449 8842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:11.633225 8827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 19:33:11.633394 8827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "b79eed4702d648bfa8dee79ecee570e1"
format_stamp: "Formatted at 2026-09-26 19:33:11 on dist-test-slave-phbh"
server_key: "398bfb4eaa0ae8ad459a58b22eb78599"
server_key_iv: "c3ee675c4ce66060e8e75b6c22e71511"
server_key_version: "encryptionkey@0"
I20260926 19:33:11.633610 8827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:11.657486 8827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:11.661872 8827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:11.662202 8827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:11.664954 8827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:11.665045 8827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:11.665220 8827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:11.665328 8827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:11 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451191, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:11.689347 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:11.665867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:51687 (local address 127.6.215.254:36299)
0926 19:33:11.668240 (+ 2373us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:11.668248 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:11.668270 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:33:11.670177 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:11.670185 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:11.670265 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:11.670403 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:11.672678 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.673777 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.676485 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.677277 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.677456 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:11.682039 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:11.682045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:11.682048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:11.682081 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:11.685480 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:11.686670 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:11.686677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:11.686681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:11.686829 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:11.687315 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:11.687323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:11.687327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:11.687623 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:11.687779 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:11.688940 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0926 19:33:11.689105 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2230,"thread_start_us":2142,"threads_started":1}
I20260926 19:33:11.691232 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:11.666556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36299 (local address 127.6.215.193:51687)
0926 19:33:11.667447 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:11.667500 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:11.669845 (+ 2345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:11.670675 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:11.670683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:11.671412 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:11.672458 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:11.672491 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.674028 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.674039 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:11.676327 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:11.676340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.677580 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.677589 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:11.677954 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:11.678849 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:11.678880 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:11.681788 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:11.685655 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:11.685663 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:11.685684 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:11.686461 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:11.686953 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:11.686961 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:11.686964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:11.687160 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:11.687875 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:11.687884 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:11.688284 (+ 400us) client_negotiation.cc:769] Sending connection context
0926 19:33:11.688613 (+ 329us) client_negotiation.cc:240] Negotiation successful
0926 19:33:11.690079 (+ 1466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":394,"threads_started":1}
I20260926 19:33:11.712110 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:11.698487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:37425 (local address 127.6.215.254:36299)
0926 19:33:11.698719 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:11.698724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:11.699604 (+ 880us) server_negotiation.cc:407] Connection header received
0926 19:33:11.699792 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:11.699797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:11.699862 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:11.699964 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:11.700994 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.702085 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.704207 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.704891 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.705065 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:11.706918 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:11.706923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:11.706926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:11.706954 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:11.709600 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:11.710247 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:11.710254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:11.710257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:11.710376 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:11.710720 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:11.710727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:11.710730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:11.710994 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:11.711143 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:11.711527 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 19:33:11.711739 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:33:11.712211 8935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:11.698575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36299 (local address 127.6.215.193:37425)
0926 19:33:11.699482 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:11.699504 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:11.699667 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:11.700090 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:11.700094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:11.700500 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:11.700826 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:11.700841 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.702277 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.702282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:11.704004 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:11.704024 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:11.705108 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:11.705114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:11.705277 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:11.705973 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:11.705989 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:11.706724 (+ 735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:11.709728 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:11.709734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:11.709738 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:11.710129 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:11.710490 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:11.710495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:11.710499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:11.710621 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:11.711110 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:11.711114 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:11.711294 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 19:33:11.711447 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 19:33:11.711761 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":538,"threads_started":1}
I20260926 19:33:11.717068 8827 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:46561
I20260926 19:33:11.717103 8962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:46561 every 8 connection(s)
I20260926 19:33:11.718336 8827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:11.720448 8963 heartbeater.cc:347] Connected to a master server at 127.6.215.254:36299
I20260926 19:33:11.720682 8963 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:11.720805 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 8827
I20260926 19:33:11.721000 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:11.721421 8963 heartbeater.cc:511] Master 127.6.215.254:36299 requested a full tablet report, sending...
I20260926 19:33:11.721688 7007 external_mini_cluster.cc:1428] Setting key 13a1d1648020c2876fb07298049dafb3
I20260926 19:33:11.723699 8756 ts_manager.cc:194] Registered new tserver with Master: b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561)
I20260926 19:33:11.726017 8756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:51687
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:11.764219 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:36299
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:12.087231 8967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:12.087497 8967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:12.087663 8967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:12.095337 8967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:12.095441 8967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:12.095587 8967 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 19:33:12.095659 8967 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:12.095794 8967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:12.105741 8967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:12.107102 8967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:12.108423 8967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:12.121804 8972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:12.121872 8975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:12.121876 8973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:12.122735 8967 server_base.cc:1034] running on GCE node
I20260926 19:33:12.123306 8967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:12.124166 8967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:12.125362 8967 hybrid_clock.cc:648] HybridClock initialized: now 1790451192125320 us; error 27 us; skew 500 ppm
I20260926 19:33:12.125640 8967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:12 dist-test-slave-phbh krb5kdc[8711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451192, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:12.134871 8967 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:12.136541 8967 webserver.cc:460] Webserver started at http://127.6.215.194:44925/ using document root <none> and password file <none>
I20260926 19:33:12.137363 8967 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:12.137530 8967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:12.137887 8967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:12.141037 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "e10440fe5ae14ab5a33921e559492c3e"
format_stamp: "Formatted at 2026-09-26 19:33:12 on dist-test-slave-phbh"
server_key: "99b57d05ea858aec17f315c1b1cedb82"
server_key_iv: "fdf8f32b0712b80c8a80ed1ae5cb06b9"
server_key_version: "encryptionkey@0"
I20260926 19:33:12.141873 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "e10440fe5ae14ab5a33921e559492c3e"
format_stamp: "Formatted at 2026-09-26 19:33:12 on dist-test-slave-phbh"
server_key: "99b57d05ea858aec17f315c1b1cedb82"
server_key_iv: "fdf8f32b0712b80c8a80ed1ae5cb06b9"
server_key_version: "encryptionkey@0"
I20260926 19:33:12.147131 8967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:12.150696 8982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:12.152079 8967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:33:12.152247 8967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "e10440fe5ae14ab5a33921e559492c3e"
format_stamp: "Formatted at 2026-09-26 19:33:12 on dist-test-slave-phbh"
server_key: "99b57d05ea858aec17f315c1b1cedb82"
server_key_iv: "fdf8f32b0712b80c8a80ed1ae5cb06b9"
server_key_version: "encryptionkey@0"
I20260926 19:33:12.152439 8967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:12.174098 8967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:12.178545 8967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:12.178859 8967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:12.181700 8967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:12.181846 8967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:12.181977 8967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:12.182073 8967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:12 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451192, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:12.208149 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.182476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:49973 (local address 127.6.215.254:36299)
0926 19:33:12.182848 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.182855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.184350 (+ 1495us) server_negotiation.cc:407] Connection header received
0926 19:33:12.186859 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.186869 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.186980 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.187100 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:12.189769 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.190754 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.194043 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.194852 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.194999 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.200253 (+ 5254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:12.200259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:12.200262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:12.200297 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:12.203451 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.204266 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.204272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.204276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.204406 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.204892 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.204898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.204902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.205270 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:12.205451 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.207560 (+ 2109us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.207789 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260926 19:33:12.208819 8991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.183164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36299 (local address 127.6.215.194:49973)
0926 19:33:12.184176 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.184229 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.186421 (+ 2192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.187462 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.187479 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.188392 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:12.189478 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.189515 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.190965 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.190972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.193847 (+ 2875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.193865 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.195169 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.195193 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.195464 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.196420 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:12.196476 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:12.199993 (+ 3517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:12.203615 (+ 3622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.203625 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.203644 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.204115 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.204524 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.204531 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.204535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.204740 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.205433 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:12.205445 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:12.206057 (+ 612us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.206517 (+ 460us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.207260 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":431,"threads_started":1}
I20260926 19:33:12.236110 8967 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:35059
I20260926 19:33:12.236162 9101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:35059 every 8 connection(s)
I20260926 19:33:12.237133 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.216578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:59541 (local address 127.6.215.254:36299)
0926 19:33:12.216815 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.216823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.219664 (+ 2841us) server_negotiation.cc:407] Connection header received
0926 19:33:12.220037 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.220041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.220107 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.220249 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:12.221521 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.222887 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.225796 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.226584 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.226784 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.229708 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:12.229717 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:12.229725 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:12.229769 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:12.233988 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.234714 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.234723 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.234730 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.234920 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.235376 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.235381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.235384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.235863 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:12.236025 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.236580 (+ 555us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.236812 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:33:12.237345 9059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.216677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36299 (local address 127.6.215.194:59541)
0926 19:33:12.219509 (+ 2832us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.219534 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.219730 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.220285 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.220290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.221051 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:12.221363 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.221372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.223096 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.223101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.225622 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.225644 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.227033 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.227039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.227171 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.227942 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:12.227977 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:12.229445 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:12.234143 (+ 4698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.234153 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.234156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.234562 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.235044 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.235054 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.235058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.235231 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.236024 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:12.236030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:12.236289 (+ 259us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.236448 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.236838 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2619,"thread_start_us":2446,"threads_started":1}
I20260926 19:33:12.237910 8967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:12.240134 9102 heartbeater.cc:347] Connected to a master server at 127.6.215.254:36299
I20260926 19:33:12.240393 9102 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:12.241145 9102 heartbeater.cc:511] Master 127.6.215.254:36299 requested a full tablet report, sending...
I20260926 19:33:12.242424 8756 ts_manager.cc:194] Registered new tserver with Master: e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:12.242801 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 8967
I20260926 19:33:12.242969 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:12.243347 8756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:49973
I20260926 19:33:12.243556 7007 external_mini_cluster.cc:1428] Setting key b39f572fc0afa0c63dd93feb9be4f1a8
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:12.284998 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:36299
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:12.610740 9106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:12.611066 9106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:12.611246 9106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:12.618688 9106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:12.618873 9106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:12.619017 9106 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 19:33:12.619102 9106 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:12.619240 9106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:12.629673 9106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:12.631151 9106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:12.632480 9106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:12.645912 9111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:12.645946 9114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:12.645881 9112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:12.646648 9106 server_base.cc:1034] running on GCE node
I20260926 19:33:12.647238 9106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:12.648133 9106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:12.649364 9106 hybrid_clock.cc:648] HybridClock initialized: now 1790451192649326 us; error 53 us; skew 500 ppm
I20260926 19:33:12.649669 9106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:12 dist-test-slave-phbh krb5kdc[8711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451192, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:12.652853 9106 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:12.654390 9106 webserver.cc:460] Webserver started at http://127.6.215.195:46339/ using document root <none> and password file <none>
I20260926 19:33:12.655040 9106 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:12.655186 9106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:12.655514 9106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:12.658489 9106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "dfa36727c8964c7eaea513e4a0601ec5"
format_stamp: "Formatted at 2026-09-26 19:33:12 on dist-test-slave-phbh"
server_key: "ee08619b9043989e7efa4d3c21c5db1d"
server_key_iv: "c60f05e295935ed87b20f0edc8a62735"
server_key_version: "encryptionkey@0"
I20260926 19:33:12.659227 9106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "dfa36727c8964c7eaea513e4a0601ec5"
format_stamp: "Formatted at 2026-09-26 19:33:12 on dist-test-slave-phbh"
server_key: "ee08619b9043989e7efa4d3c21c5db1d"
server_key_iv: "c60f05e295935ed87b20f0edc8a62735"
server_key_version: "encryptionkey@0"
I20260926 19:33:12.664245 9106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260926 19:33:12.667726 9121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:12.669104 9106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:12.669340 9106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "dfa36727c8964c7eaea513e4a0601ec5"
format_stamp: "Formatted at 2026-09-26 19:33:12 on dist-test-slave-phbh"
server_key: "ee08619b9043989e7efa4d3c21c5db1d"
server_key_iv: "c60f05e295935ed87b20f0edc8a62735"
server_key_version: "encryptionkey@0"
I20260926 19:33:12.669591 9106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:12.692021 9106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:12.696586 9106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:12.696949 9106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:12.699771 9106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:12.699922 9106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:12.700047 9106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:12.700157 9106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:12 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451192, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:12.725078 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.700235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:41877 (local address 127.6.215.254:36299)
0926 19:33:12.700453 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.700463 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.702127 (+ 1664us) server_negotiation.cc:407] Connection header received
0926 19:33:12.704265 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.704273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.704379 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.704503 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:12.706755 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.708087 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.711286 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.711995 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.712148 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.717082 (+ 4934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:12.717091 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:12.717098 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:12.717161 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:12.721552 (+ 4391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.722284 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.722293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.722299 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.722494 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.722975 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.722982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.722988 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.723426 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:12.723613 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.724287 (+ 674us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.724570 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 19:33:12.726094 9130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.701080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36299 (local address 127.6.215.195:41877)
0926 19:33:12.701945 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.701992 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.703836 (+ 1844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.704793 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.704808 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.705620 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:12.706470 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.706492 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.708320 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.708325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.711086 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.711110 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.712258 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.712267 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.712503 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.713268 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:12.713306 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:12.716734 (+ 3428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:12.721715 (+ 4981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.721724 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.721738 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.722141 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.722642 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.722647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.722650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.722854 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.723568 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:12.723573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:12.723893 (+ 320us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.724097 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.724467 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":370,"threads_started":1}
I20260926 19:33:12.728987 8963 heartbeater.cc:503] Master 127.6.215.254:36299 was elected leader, sending a full tablet report...
I20260926 19:33:12.752305 9106 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:38147
I20260926 19:33:12.752395 9240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:38147 every 8 connection(s)
I20260926 19:33:12.753989 9106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:12.754729 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.736986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:45879 (local address 127.6.215.254:36299)
0926 19:33:12.737301 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.737311 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.738441 (+ 1130us) server_negotiation.cc:407] Connection header received
0926 19:33:12.738671 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.738687 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.738775 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.738895 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:12.740390 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.741701 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.744075 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.744822 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.744968 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.747821 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:12.747831 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:12.747839 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:12.747881 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:12.751713 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.752662 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.752668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.752671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.752836 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.753209 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.753214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.753217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.753571 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:12.753705 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.754087 (+ 382us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.754315 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 19:33:12.754808 9199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.737606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36299 (local address 127.6.215.195:45879)
0926 19:33:12.738318 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.738340 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.738508 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.738955 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.738962 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.739679 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:12.740189 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.740207 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.741880 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.741887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.743922 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.743934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.745049 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.745058 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.745280 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.746140 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:12.746168 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:12.747579 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:12.751860 (+ 4281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.751868 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.751873 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.752557 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.752948 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:12.752952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:12.752954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:12.753073 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:12.753691 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:12.753694 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:12.753848 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.753956 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.754273 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":354,"threads_started":1}
I20260926 19:33:12.756564 9241 heartbeater.cc:347] Connected to a master server at 127.6.215.254:36299
I20260926 19:33:12.756834 9241 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:12.757609 9241 heartbeater.cc:511] Master 127.6.215.254:36299 requested a full tablet report, sending...
I20260926 19:33:12.759023 8756 ts_manager.cc:194] Registered new tserver with Master: dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195:38147)
I20260926 19:33:12.759850 8756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:41877
I20260926 19:33:12.763617 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 9106
I20260926 19:33:12.763751 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:12.764138 7007 external_mini_cluster.cc:1428] Setting key c4224bb1ba69b2b454d067160beff137
I20260926 19:33:12.766938 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 19:33:12 dist-test-slave-phbh krb5kdc[8711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 19:33:12 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:12.813781 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.799321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.6.215.254:36299)
0926 19:33:12.799522 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.799527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.799851 (+ 324us) server_negotiation.cc:407] Connection header received
0926 19:33:12.800003 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.800007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.800071 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.800203 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:12.802129 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.802903 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.804090 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.804557 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.804653 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.808475 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:12.808480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:12.808483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:12.808515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:12.811292 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.811983 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.811987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.811990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.812114 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:12.812469 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:12.812474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:12.812476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:12.812849 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:12.813037 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.813256 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.813417 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 19:33:12.818091 8756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:12.825153 8756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:12.854691 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.837382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:35059 (local address 127.0.0.1:43694)
0926 19:33:12.845788 (+ 8406us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.845812 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.845987 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.847350 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.847358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.847384 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:12.847709 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.847722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.849540 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.849544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.851876 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.851888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.853593 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.853600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.853797 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.853869 (+ 72us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.854004 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.854218 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8247,"mutex_wait_us":143,"thread_start_us":8824,"threads_started":1}
I20260926 19:33:12.855088 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.837396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.6.215.193:46561)
0926 19:33:12.847066 (+ 9670us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.847074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.847101 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:33:12.847301 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.847309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.847572 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.847755 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:12.848398 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.849526 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.852345 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.853840 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.854307 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.854551 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.854631 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.854794 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9214,"spinlock_wait_cycles":603904,"thread_start_us":9091,"threads_started":1}
I20260926 19:33:12.855692 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.838018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43694 (local address 127.6.215.194:35059)
0926 19:33:12.845317 (+ 7299us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.845331 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.845946 (+ 615us) server_negotiation.cc:407] Connection header received
0926 19:33:12.846772 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.846783 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.847096 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.847294 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:12.847923 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.849404 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.852026 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.853385 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.853538 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.853753 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.855127 (+ 1374us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.855393 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7096,"thread_start_us":6962,"threads_started":1}
I20260926 19:33:12.855854 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.838549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33886 (local address 127.6.215.195:38147)
0926 19:33:12.839327 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.839339 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.846633 (+ 7294us) server_negotiation.cc:407] Connection header received
0926 19:33:12.847332 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.847339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.847548 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.847755 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:12.848623 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.850215 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.852661 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.853792 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.853925 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.854156 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.854776 (+ 620us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.854933 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":342,"threads_started":1}
I20260926 19:33:12.855028 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.837262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:46561 (local address 127.0.0.1:33448)
0926 19:33:12.838119 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.838189 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.838380 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.847753 (+ 9373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.847763 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.847788 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:12.848223 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.848234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.849669 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.849678 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.852178 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.852192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.854084 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.854089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.854354 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.854389 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.854467 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.854594 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":54,"thread_start_us":426,"threads_started":1}
I20260926 19:33:12.855257 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.837984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:38147 (local address 127.0.0.1:33886)
0926 19:33:12.846513 (+ 8529us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.846531 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.847159 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.847989 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.847997 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.848020 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:12.848406 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.848416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.850387 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.850393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.852518 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.852527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.854000 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.854007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.854210 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.854287 (+ 77us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.854378 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.854499 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8383,"mutex_wait_us":28,"thread_start_us":7523,"threads_started":1}
I20260926 19:33:12.860137 8893 tablet_service.cc:1467] Processing CreateTablet for tablet 762f50cb8fca4432837735c3e9b3f8df (DEFAULT_TABLE table=test-table [id=81179012b605484eb4ac6f5a2f010d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:12.860137 9032 tablet_service.cc:1467] Processing CreateTablet for tablet 762f50cb8fca4432837735c3e9b3f8df (DEFAULT_TABLE table=test-table [id=81179012b605484eb4ac6f5a2f010d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:12.861773 9032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762f50cb8fca4432837735c3e9b3f8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:12.862329 8893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762f50cb8fca4432837735c3e9b3f8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:12.863507 9171 tablet_service.cc:1467] Processing CreateTablet for tablet 762f50cb8fca4432837735c3e9b3f8df (DEFAULT_TABLE table=test-table [id=81179012b605484eb4ac6f5a2f010d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:12.864732 9171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 762f50cb8fca4432837735c3e9b3f8df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:12.880678 9260 tablet_bootstrap.cc:492] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Bootstrap starting.
I20260926 19:33:12.882965 9262 tablet_bootstrap.cc:492] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Bootstrap starting.
I20260926 19:33:12.883126 9261 tablet_bootstrap.cc:492] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Bootstrap starting.
I20260926 19:33:12.884745 9260 tablet_bootstrap.cc:654] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:12.886068 9260 log.cc:824] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:12.886551 9262 tablet_bootstrap.cc:654] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:12.886973 9261 tablet_bootstrap.cc:654] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:12.887773 9262 log.cc:824] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:12.888010 9261 log.cc:824] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:12.889426 9260 tablet_bootstrap.cc:492] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: No bootstrap required, opened a new log
I20260926 19:33:12.889698 9260 ts_tablet_manager.cc:1397] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260926 19:33:12.890782 9261 tablet_bootstrap.cc:492] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: No bootstrap required, opened a new log
I20260926 19:33:12.890782 9262 tablet_bootstrap.cc:492] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: No bootstrap required, opened a new log
I20260926 19:33:12.891041 9262 ts_tablet_manager.cc:1397] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260926 19:33:12.891041 9261 ts_tablet_manager.cc:1397] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 19:33:12.899581 9260 raft_consensus.cc:348] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.900192 9260 raft_consensus.cc:374] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:12.900332 9260 raft_consensus.cc:729] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Initialized, Role: FOLLOWER
I20260926 19:33:12.901219 9260 consensus_queue.cc:260] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.902124 9261 raft_consensus.cc:348] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.902577 9261 raft_consensus.cc:374] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:12.902683 9261 raft_consensus.cc:729] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10440fe5ae14ab5a33921e559492c3e, State: Initialized, Role: FOLLOWER
I20260926 19:33:12.903352 9261 consensus_queue.cc:260] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.903885 9262 raft_consensus.cc:348] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.904472 9262 raft_consensus.cc:374] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:12.904635 9262 raft_consensus.cc:729] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79eed4702d648bfa8dee79ecee570e1, State: Initialized, Role: FOLLOWER
I20260926 19:33:12.905395 9262 consensus_queue.cc:260] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.908454 9260 ts_tablet_manager.cc:1428] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
I20260926 19:33:12.908881 9241 heartbeater.cc:503] Master 127.6.215.254:36299 was elected leader, sending a full tablet report...
I20260926 19:33:12.909454 9262 ts_tablet_manager.cc:1428] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260926 19:33:12.911544 9261 ts_tablet_manager.cc:1428] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Time spent starting tablet: real 0.020s user 0.014s sys 0.004s
I20260926 19:33:12.911710 9102 heartbeater.cc:503] Master 127.6.215.254:36299 was elected leader, sending a full tablet report...
I20260926 19:33:12.930708 9267 raft_consensus.cc:482] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:12.931062 9267 raft_consensus.cc:504] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.932662 9267 leader_election.cc:288] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195:38147)
I20260926 19:33:12.941901 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.933256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:56857 (local address 127.6.215.195:38147)
0926 19:33:12.933481 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.933489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.933997 (+ 508us) server_negotiation.cc:407] Connection header received
0926 19:33:12.934211 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.934217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.934297 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.934403 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:12.935106 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.936263 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.939495 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.940728 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.940926 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.940992 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.941502 (+ 510us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.941639 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 19:33:12.942082 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.933002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:38147 (local address 127.6.215.194:56857)
0926 19:33:12.933881 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.933901 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.934062 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.934447 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.934453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.934479 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:12.934948 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.934966 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.936427 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.936433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.939307 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.939322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.941168 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.941177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.941306 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.941365 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.941441 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.941596 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":226,"thread_start_us":484,"threads_started":1}
I20260926 19:33:12.942166 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.933137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:46561 (local address 127.6.215.194:58113)
0926 19:33:12.933971 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.933987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.934085 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.934474 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.934477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.934493 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:12.935045 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.935058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.936429 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.936433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.939307 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.939322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.941189 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.941194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.941364 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.941388 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 19:33:12.941565 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 19:33:12.941725 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":258,"thread_start_us":373,"threads_started":1}
I20260926 19:33:12.942385 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.933086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:58113 (local address 127.6.215.193:46561)
0926 19:33:12.933293 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.933300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.934080 (+ 780us) server_negotiation.cc:407] Connection header received
0926 19:33:12.934265 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.934273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.934368 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.934456 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:12.935205 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.936309 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.939495 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.940730 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.940919 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:12.940992 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:12.941587 (+ 595us) server_negotiation.cc:299] Negotiation successful
0926 19:33:12.941717 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 19:33:12.943188 9195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762f50cb8fca4432837735c3e9b3f8df" candidate_uuid: "e10440fe5ae14ab5a33921e559492c3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfa36727c8964c7eaea513e4a0601ec5" is_pre_election: true
I20260926 19:33:12.943329 8913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762f50cb8fca4432837735c3e9b3f8df" candidate_uuid: "e10440fe5ae14ab5a33921e559492c3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1" is_pre_election: true
I20260926 19:33:12.943720 9195 raft_consensus.cc:2454] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10440fe5ae14ab5a33921e559492c3e in term 0.
I20260926 19:33:12.943799 8913 raft_consensus.cc:2454] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10440fe5ae14ab5a33921e559492c3e in term 0.
I20260926 19:33:12.944430 8986 leader_election.cc:302] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b79eed4702d648bfa8dee79ecee570e1, dfa36727c8964c7eaea513e4a0601ec5, e10440fe5ae14ab5a33921e559492c3e; no voters:
I20260926 19:33:12.944865 9267 raft_consensus.cc:2781] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:12.945011 9267 raft_consensus.cc:482] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:12.945111 9267 raft_consensus.cc:3037] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:12.947160 9267 raft_consensus.cc:504] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.947753 9267 leader_election.cc:288] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [CANDIDATE]: Term 1 election: Requested vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195:38147)
I20260926 19:33:12.948184 8913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762f50cb8fca4432837735c3e9b3f8df" candidate_uuid: "e10440fe5ae14ab5a33921e559492c3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1"
I20260926 19:33:12.948287 9195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "762f50cb8fca4432837735c3e9b3f8df" candidate_uuid: "e10440fe5ae14ab5a33921e559492c3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfa36727c8964c7eaea513e4a0601ec5"
I20260926 19:33:12.948375 8913 raft_consensus.cc:3037] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:12.948529 9195 raft_consensus.cc:3037] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:12.950572 8913 raft_consensus.cc:2454] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10440fe5ae14ab5a33921e559492c3e in term 1.
I20260926 19:33:12.950958 8986 leader_election.cc:302] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79eed4702d648bfa8dee79ecee570e1, e10440fe5ae14ab5a33921e559492c3e; no voters:
I20260926 19:33:12.951186 9195 raft_consensus.cc:2454] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10440fe5ae14ab5a33921e559492c3e in term 1.
I20260926 19:33:12.951277 9267 raft_consensus.cc:2781] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:12.951753 9267 raft_consensus.cc:686] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 1 LEADER]: Becoming Leader. State: Replica: e10440fe5ae14ab5a33921e559492c3e, State: Running, Role: LEADER
I20260926 19:33:12.952240 9267 consensus_queue.cc:237] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:12.957165 8754 catalog_manager.cc:5495] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e reported cstate change: term changed from 0 to 1, leader changed from <none> to e10440fe5ae14ab5a33921e559492c3e (127.6.215.194). New cstate: current_term: 1 leader_uuid: "e10440fe5ae14ab5a33921e559492c3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } health_report { overall_health: UNKNOWN } } }
W20260926 19:33:12.970675 8964 tablet.cc:2369] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:33:12.990113 9103 tablet.cc:2369] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 19:33:12 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451191, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
W20260926 19:33:13.006247 9242 tablet.cc:2369] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:13.009334 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.995360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36299 (local address 127.0.0.1:43120)
0926 19:33:12.996001 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:12.996022 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:12.996201 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:12.996522 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:12.996525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:12.997329 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:12.997579 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.997587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.998708 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.998712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:12.999650 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:12.999660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.000453 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.000457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.000563 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.001247 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:13.001269 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:13.003473 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:13.006749 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:13.006762 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:13.006770 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:13.007174 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:13.007571 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:13.007576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:13.007580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:13.007735 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:13.008406 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:13.008412 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:13.008553 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 19:33:13.008691 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 19:33:13.008970 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":369,"threads_started":1}
I20260926 19:33:13.009452 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:12.995219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43120 (local address 127.6.215.254:36299)
0926 19:33:12.995439 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:12.995446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:12.996178 (+ 732us) server_negotiation.cc:407] Connection header received
0926 19:33:12.996312 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:12.996316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:12.996391 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:12.996481 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:12.997733 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:12.998587 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:12.999776 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.000287 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.000388 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.003692 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:13.003697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:13.003700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:13.003730 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:13.006605 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:13.007309 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:13.007314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:13.007318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:13.007449 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:13.007844 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:13.007852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:13.007857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:13.008272 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:13.008422 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.008869 (+ 447us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.009107 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":80}
I20260926 19:33:13.014384 8754 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43120:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 19:33:13.015477 8754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:13.022396 8893 tablet_service.cc:1467] Processing CreateTablet for tablet c05f52f12dd4460d9ca05aaeee710762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0b9e844fd8b245c49ee4d4a9087dcc1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:33:13.022688 9032 tablet_service.cc:1467] Processing CreateTablet for tablet c05f52f12dd4460d9ca05aaeee710762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0b9e844fd8b245c49ee4d4a9087dcc1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:33:13.022693 9171 tablet_service.cc:1467] Processing CreateTablet for tablet c05f52f12dd4460d9ca05aaeee710762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0b9e844fd8b245c49ee4d4a9087dcc1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:33:13.023442 8893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05f52f12dd4460d9ca05aaeee710762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:13.023439 9032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05f52f12dd4460d9ca05aaeee710762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:13.023448 9171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05f52f12dd4460d9ca05aaeee710762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:13.029690 9260 tablet_bootstrap.cc:492] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: Bootstrap starting.
I20260926 19:33:13.030681 9262 tablet_bootstrap.cc:492] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: Bootstrap starting.
I20260926 19:33:13.030851 9261 tablet_bootstrap.cc:492] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: Bootstrap starting.
I20260926 19:33:13.031252 9260 tablet_bootstrap.cc:654] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:13.032055 9261 tablet_bootstrap.cc:654] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:13.032351 9262 tablet_bootstrap.cc:654] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:13.033988 9260 tablet_bootstrap.cc:492] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: No bootstrap required, opened a new log
I20260926 19:33:13.034168 9260 ts_tablet_manager.cc:1397] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260926 19:33:13.034392 9261 tablet_bootstrap.cc:492] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: No bootstrap required, opened a new log
I20260926 19:33:13.034539 9261 ts_tablet_manager.cc:1397] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 19:33:13.035159 9261 raft_consensus.cc:348] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.035374 9261 raft_consensus.cc:374] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:13.035180 9260 raft_consensus.cc:348] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.035473 9261 raft_consensus.cc:729] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10440fe5ae14ab5a33921e559492c3e, State: Initialized, Role: FOLLOWER
I20260926 19:33:13.035481 9260 raft_consensus.cc:374] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:13.035583 9260 raft_consensus.cc:729] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Initialized, Role: FOLLOWER
I20260926 19:33:13.035763 9261 consensus_queue.cc:260] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.035871 9260 consensus_queue.cc:260] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.036164 9267 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.036429 9267 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:13.036578 9266 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.037009 9266 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:13.037072 9261 ts_tablet_manager.cc:1428] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 19:33:13.038674 9260 ts_tablet_manager.cc:1428] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260926 19:33:13.040395 9262 tablet_bootstrap.cc:492] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: No bootstrap required, opened a new log
I20260926 19:33:13.040627 9262 ts_tablet_manager.cc:1397] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20260926 19:33:13.041446 9262 raft_consensus.cc:348] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.041647 9262 raft_consensus.cc:374] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:13.041741 9262 raft_consensus.cc:729] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79eed4702d648bfa8dee79ecee570e1, State: Initialized, Role: FOLLOWER
I20260926 19:33:13.041901 9262 consensus_queue.cc:260] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.042484 9262 ts_tablet_manager.cc:1428] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 19:33:13.042305 9268 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.042766 9268 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:13.097962 9266 raft_consensus.cc:482] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:13.098237 9266 raft_consensus.cc:504] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.099843 9266 leader_election.cc:288] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:13.105918 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.100353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:43027 (local address 127.6.215.193:46561)
0926 19:33:13.100540 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.100547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.100570 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:33:13.100645 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.100649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.100705 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.100854 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:13.101245 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.102199 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.104247 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.105036 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.105214 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.105291 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.105542 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.105660 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:33:13.105916 9130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.100217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:46561 (local address 127.6.215.195:43027)
0926 19:33:13.100412 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:13.100434 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:13.100550 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:13.100882 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:13.100886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:13.100912 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:13.101103 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.101111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.102349 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.102378 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.104096 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.104107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.105194 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.105199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.105388 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.105411 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 19:33:13.105517 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 19:33:13.105643 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 19:33:13.106727 8913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f52f12dd4460d9ca05aaeee710762" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1" is_pre_election: true
I20260926 19:33:13.106958 9282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.100237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:35059 (local address 127.6.215.195:49023)
0926 19:33:13.101446 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:13.101461 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:13.101556 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:13.101861 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:13.101864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:13.101886 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:13.102254 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.102263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.103639 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.103643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.105106 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.105117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.106274 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.106282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.106471 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.106509 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 19:33:13.106590 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 19:33:13.106667 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":376,"thread_start_us":484,"threads_started":1}
I20260926 19:33:13.107004 8913 raft_consensus.cc:2454] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:13.107059 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.100360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:49023 (local address 127.6.215.194:35059)
0926 19:33:13.100553 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.100561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.101555 (+ 994us) server_negotiation.cc:407] Connection header received
0926 19:33:13.101642 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.101647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.101718 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.101812 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:13.102434 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.103495 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.105270 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.106076 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.106239 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.106304 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.106682 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.106817 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:33:13.107632 9125 leader_election.cc:302] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79eed4702d648bfa8dee79ecee570e1, dfa36727c8964c7eaea513e4a0601ec5; no voters:
I20260926 19:33:13.108068 9266 raft_consensus.cc:2781] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:13.107951 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f52f12dd4460d9ca05aaeee710762" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e" is_pre_election: true
I20260926 19:33:13.108213 9266 raft_consensus.cc:482] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:13.108335 9266 raft_consensus.cc:3037] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:13.108372 9056 raft_consensus.cc:2454] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:13.111076 9266 raft_consensus.cc:504] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.111677 9266 leader_election.cc:288] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Requested vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:13.112142 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f52f12dd4460d9ca05aaeee710762" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e"
I20260926 19:33:13.112123 8913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f52f12dd4460d9ca05aaeee710762" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1"
I20260926 19:33:13.112331 9056 raft_consensus.cc:3037] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:13.112375 8913 raft_consensus.cc:3037] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:13.115355 8913 raft_consensus.cc:2454] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:13.115370 9056 raft_consensus.cc:2454] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:13.115779 9125 leader_election.cc:302] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79eed4702d648bfa8dee79ecee570e1, dfa36727c8964c7eaea513e4a0601ec5; no voters:
I20260926 19:33:13.116096 9266 raft_consensus.cc:2781] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:13.116566 9266 raft_consensus.cc:686] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 LEADER]: Becoming Leader. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Running, Role: LEADER
I20260926 19:33:13.117116 9266 consensus_queue.cc:237] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.120270 9283 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: TxnStatusTablet state changed. Reason: New leader dfa36727c8964c7eaea513e4a0601ec5. Latest consensus state: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.120667 9283 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:33:13.121421 9285 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 19:33:13.121685 9285 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 19:33:13.121562 8754 catalog_manager.cc:5495] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } health_report { overall_health: HEALTHY } } }
I20260926 19:33:13.145262 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.138590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33890 (local address 127.6.215.195:38147)
0926 19:33:13.138756 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.138762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.138783 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:33:13.138963 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.138969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.139078 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.139230 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:13.139642 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.140910 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.142861 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.143299 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.143414 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.143968 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.144543 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.144650 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.144890 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.144963 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 19:33:13.145284 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.138405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:38147 (local address 127.0.0.1:33890)
0926 19:33:13.138630 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:13.138650 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:13.138777 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:13.139325 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:13.139329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:13.139339 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:33:13.139492 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.139501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.141070 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.141075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.142716 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.142731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.143455 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.143462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.143690 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.143784 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.144660 (+ 876us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.144681 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 19:33:13.144812 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 19:33:13.144971 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 19:33:13.153738 9056 raft_consensus.cc:1261] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:13.153762 8913 raft_consensus.cc:1261] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:13.154716 9283 consensus_queue.cc:1035] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:13.155043 9266 consensus_queue.cc:1035] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:13.166491 9267 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: TxnStatusTablet state changed. Reason: New leader dfa36727c8964c7eaea513e4a0601ec5. Latest consensus state: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.166894 9267 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:13.168115 9268 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: TxnStatusTablet state changed. Reason: New leader dfa36727c8964c7eaea513e4a0601ec5. Latest consensus state: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.168411 9268 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:13.170015 9288 mvcc.cc:205] Tried to move back new op lower bound from 7333688087148019712 to 7333688087011209216. Current Snapshot: MvccSnapshot[applied={T|T < 7333688087148019712}]
I20260926 19:33:13.171685 9267 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.171965 9267 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P e10440fe5ae14ab5a33921e559492c3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:13.175467 9268 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.175725 9268 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P b79eed4702d648bfa8dee79ecee570e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:33:13.170529 9266 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.181106 9266 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:33:13.183916 9266 tablet_replica.cc:439] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } } }
I20260926 19:33:13.184161 9266 tablet_replica.cc:442] T c05f52f12dd4460d9ca05aaeee710762 P dfa36727c8964c7eaea513e4a0601ec5: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:33:13.195894 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.190230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.6.215.194:35059)
0926 19:33:13.190402 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.190410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.190487 (+ 77us) server_negotiation.cc:407] Connection header received
0926 19:33:13.190748 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.190760 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.190837 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.190994 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:13.191364 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.192094 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.193487 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.194078 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.194207 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.194640 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.195235 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.195420 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.195476 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.195553 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 19:33:13.212332 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.206758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:33203 (local address 127.6.215.195:38147)
0926 19:33:13.206994 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.207002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.207357 (+ 355us) server_negotiation.cc:407] Connection header received
0926 19:33:13.207552 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.207560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.207627 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.207721 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:13.208246 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.209082 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.210437 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.210928 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.211070 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.211537 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.211674 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.211777 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.211926 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.211999 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 19:33:13.212399 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.206651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:38147 (local address 127.6.215.194:33203)
0926 19:33:13.207236 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:13.207251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:13.207415 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:13.207735 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:13.207739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:13.207750 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:33:13.208094 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.208102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.209228 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.209235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.210280 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.210291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.211110 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.211119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.211248 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.211297 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.211789 (+ 492us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.211802 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 19:33:13.211903 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 19:33:13.212034 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":268,"threads_started":1}
I20260926 19:33:13.222529 8913 raft_consensus.cc:1261] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Refusing update from remote peer e10440fe5ae14ab5a33921e559492c3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:13.222584 9195 raft_consensus.cc:1261] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Refusing update from remote peer e10440fe5ae14ab5a33921e559492c3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:13.223313 9267 consensus_queue.cc:1035] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:13.223603 9271 consensus_queue.cc:1035] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:13.231587 9289 mvcc.cc:205] Tried to move back new op lower bound from 7333688087433359360 to 7333688086336344064. Current Snapshot: MvccSnapshot[applied={T|T < 7333688087433359360}]
I20260926 19:33:13.287356 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.268850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:53219 (local address 127.6.215.194:35059)
0926 19:33:13.269100 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.269110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.280907 (+ 11797us) server_negotiation.cc:407] Connection header received
0926 19:33:13.281141 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.281149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.281228 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.281346 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:13.281935 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.283063 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.285280 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.285885 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.286021 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.286571 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.286670 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.286776 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.286985 (+ 209us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.287062 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 19:33:13.287516 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.266093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:35059 (local address 127.6.215.195:53219)
0926 19:33:13.280738 (+ 14645us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:13.280755 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:13.280989 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:13.281362 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:13.281368 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:13.281381 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:33:13.281722 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.281739 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.283242 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.283247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.285115 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.285132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.286085 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.286094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.286299 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.286361 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.286828 (+ 467us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.286840 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 19:33:13.286960 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 19:33:13.287100 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14451,"thread_start_us":14320,"threads_started":1}
I20260926 19:33:13.330181 8756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:13.330478 8756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:13.334578 8756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Sending DeleteTablet for 3 replicas of tablet 762f50cb8fca4432837735c3e9b3f8df
I20260926 19:33:13.335755 9032 tablet_service.cc:1514] Processing DeleteTablet for tablet 762f50cb8fca4432837735c3e9b3f8df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:13 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43694
I20260926 19:33:13.336107 9171 tablet_service.cc:1514] Processing DeleteTablet for tablet 762f50cb8fca4432837735c3e9b3f8df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:13 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33886
I20260926 19:33:13.336828 8893 tablet_service.cc:1514] Processing DeleteTablet for tablet 762f50cb8fca4432837735c3e9b3f8df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:13 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33448
I20260926 19:33:13.344111 8756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:13.345640 8756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:13.346123 9313 tablet_replica.cc:330] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: stopping tablet replica
I20260926 19:33:13.346576 9313 raft_consensus.cc:2229] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:13.346967 9313 raft_consensus.cc:2258] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:13.354310 9032 tablet_service.cc:1467] Processing CreateTablet for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 (DEFAULT_TABLE table=test-table [id=18a418c40c0544fab760521221d4d89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:13.354810 9311 tablet_replica.cc:330] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: stopping tablet replica
I20260926 19:33:13.354725 9170 tablet_service.cc:1467] Processing CreateTablet for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 (DEFAULT_TABLE table=test-table [id=18a418c40c0544fab760521221d4d89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:13.355337 9170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:13.356112 9032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:13.361212 8893 tablet_service.cc:1467] Processing CreateTablet for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 (DEFAULT_TABLE table=test-table [id=18a418c40c0544fab760521221d4d89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:13.361754 8893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:13.362653 9311 raft_consensus.cc:2229] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:13.363360 9311 raft_consensus.cc:2258] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:13.368386 9311 ts_tablet_manager.cc:1905] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:13.369618 9260 tablet_bootstrap.cc:492] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: Bootstrap starting.
I20260926 19:33:13.371853 9260 tablet_bootstrap.cc:654] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:13.372892 9261 tablet_bootstrap.cc:492] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: Bootstrap starting.
I20260926 19:33:13.374735 9262 tablet_bootstrap.cc:492] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: Bootstrap starting.
I20260926 19:33:13.375005 9261 tablet_bootstrap.cc:654] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:13.376547 9262 tablet_bootstrap.cc:654] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:13.380714 9311 ts_tablet_manager.cc:1918] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:13.380924 9311 log.cc:1196] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/762f50cb8fca4432837735c3e9b3f8df
I20260926 19:33:13.381592 9311 ts_tablet_manager.cc:1939] T 762f50cb8fca4432837735c3e9b3f8df P e10440fe5ae14ab5a33921e559492c3e: Deleting consensus metadata
I20260926 19:33:13.383486 8741 catalog_manager.cc:4841] TS e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059): tablet 762f50cb8fca4432837735c3e9b3f8df (table test-table [id=81179012b605484eb4ac6f5a2f010d1b]) successfully deleted
I20260926 19:33:13.386862 9312 tablet_replica.cc:330] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: stopping tablet replica
I20260926 19:33:13.386843 9262 tablet_bootstrap.cc:492] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: No bootstrap required, opened a new log
I20260926 19:33:13.387342 9262 ts_tablet_manager.cc:1397] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.007s
I20260926 19:33:13.387344 9312 raft_consensus.cc:2229] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:13.387701 9312 raft_consensus.cc:2258] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:13.389496 9260 tablet_bootstrap.cc:492] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: No bootstrap required, opened a new log
I20260926 19:33:13.389601 9312 ts_tablet_manager.cc:1905] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:13.389675 9260 ts_tablet_manager.cc:1397] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.006s
I20260926 19:33:13.389845 9313 ts_tablet_manager.cc:1905] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:13.390429 9260 raft_consensus.cc:348] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.390661 9260 raft_consensus.cc:374] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:13.390763 9260 raft_consensus.cc:729] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Initialized, Role: FOLLOWER
I20260926 19:33:13.391073 9260 consensus_queue.cc:260] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.390355 9262 raft_consensus.cc:348] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.391480 9262 raft_consensus.cc:374] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:13.391779 9260 ts_tablet_manager.cc:1428] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 19:33:13.391914 9262 raft_consensus.cc:729] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79eed4702d648bfa8dee79ecee570e1, State: Initialized, Role: FOLLOWER
I20260926 19:33:13.392354 9262 consensus_queue.cc:260] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.395509 9262 ts_tablet_manager.cc:1428] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260926 19:33:13.398785 9312 ts_tablet_manager.cc:1918] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:13.398948 9312 log.cc:1196] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/762f50cb8fca4432837735c3e9b3f8df
I20260926 19:33:13.399490 9312 ts_tablet_manager.cc:1939] T 762f50cb8fca4432837735c3e9b3f8df P dfa36727c8964c7eaea513e4a0601ec5: Deleting consensus metadata
I20260926 19:33:13.399842 9261 tablet_bootstrap.cc:492] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: No bootstrap required, opened a new log
I20260926 19:33:13.400038 9261 ts_tablet_manager.cc:1397] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.007s
I20260926 19:33:13.400871 9261 raft_consensus.cc:348] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.401134 9261 raft_consensus.cc:374] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:13.401185 8742 catalog_manager.cc:4841] TS dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195:38147): tablet 762f50cb8fca4432837735c3e9b3f8df (table test-table [id=81179012b605484eb4ac6f5a2f010d1b]) successfully deleted
I20260926 19:33:13.401232 9261 raft_consensus.cc:729] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10440fe5ae14ab5a33921e559492c3e, State: Initialized, Role: FOLLOWER
I20260926 19:33:13.401517 9261 consensus_queue.cc:260] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.404639 9313 ts_tablet_manager.cc:1918] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:13.404904 9313 log.cc:1196] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/762f50cb8fca4432837735c3e9b3f8df
I20260926 19:33:13.404937 9261 ts_tablet_manager.cc:1428] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260926 19:33:13.405521 9313 ts_tablet_manager.cc:1939] T 762f50cb8fca4432837735c3e9b3f8df P b79eed4702d648bfa8dee79ecee570e1: Deleting consensus metadata
I20260926 19:33:13.410797 8743 catalog_manager.cc:4841] TS b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561): tablet 762f50cb8fca4432837735c3e9b3f8df (table test-table [id=81179012b605484eb4ac6f5a2f010d1b]) successfully deleted
I20260926 19:33:13.623107 9294 raft_consensus.cc:482] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:13.623293 9294 raft_consensus.cc:504] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.623977 9294 leader_election.cc:288] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:13.624532 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ed5c1b53dd417cbbcb0d6e8e27c5f0" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e" is_pre_election: true
I20260926 19:33:13.624588 8913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ed5c1b53dd417cbbcb0d6e8e27c5f0" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1" is_pre_election: true
I20260926 19:33:13.624811 9056 raft_consensus.cc:2454] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:13.624890 8913 raft_consensus.cc:2454] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:13.625298 9123 leader_election.cc:302] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfa36727c8964c7eaea513e4a0601ec5, e10440fe5ae14ab5a33921e559492c3e; no voters:
I20260926 19:33:13.625582 9294 raft_consensus.cc:2781] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:13.625705 9294 raft_consensus.cc:482] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:13.625793 9294 raft_consensus.cc:3037] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:13.628911 9294 raft_consensus.cc:504] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.630120 9294 leader_election.cc:288] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Requested vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:13.630664 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ed5c1b53dd417cbbcb0d6e8e27c5f0" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e"
I20260926 19:33:13.630918 9056 raft_consensus.cc:3037] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:13.633008 8913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86ed5c1b53dd417cbbcb0d6e8e27c5f0" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1"
I20260926 19:33:13.633221 8913 raft_consensus.cc:3037] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:13.634531 9056 raft_consensus.cc:2454] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:13.634964 9123 leader_election.cc:302] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfa36727c8964c7eaea513e4a0601ec5, e10440fe5ae14ab5a33921e559492c3e; no voters:
I20260926 19:33:13.635267 9294 raft_consensus.cc:2781] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:13.635509 8913 raft_consensus.cc:2454] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:13.635538 9294 raft_consensus.cc:686] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 LEADER]: Becoming Leader. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Running, Role: LEADER
I20260926 19:33:13.636013 9294 consensus_queue.cc:237] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:13.638813 8754 catalog_manager.cc:5495] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } health_report { overall_health: HEALTHY } } }
I20260926 19:33:13.666013 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.661195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.6.215.195:38147)
0926 19:33:13.661371 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.661378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.661473 (+ 95us) server_negotiation.cc:407] Connection header received
0926 19:33:13.661671 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.661677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.661751 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.661853 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:13.662202 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.662925 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.664246 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.664695 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.664874 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.665278 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.665410 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.665513 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.665659 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.665733 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 19:33:13.679399 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.674105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:34597 (local address 127.6.215.195:38147)
0926 19:33:13.674253 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:13.674257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:13.674304 (+ 47us) server_negotiation.cc:407] Connection header received
0926 19:33:13.674481 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:13.674485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:13.674541 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:13.674614 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:13.675003 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.675681 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.677482 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.677850 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.677937 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.678412 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.678497 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.678557 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:13.678720 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 19:33:13.678809 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":47}
I20260926 19:33:13.679657 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:13.674050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:38147 (local address 127.6.215.195:34597)
0926 19:33:13.674229 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:13.674255 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:13.674371 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:13.674666 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:13.674671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:13.674683 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:33:13.674874 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.674888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.675822 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.675828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.677351 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:13.677366 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:13.677995 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:13.678004 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:13.678195 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:13.678255 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:33:13.678596 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:33:13.678608 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 19:33:13.678699 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 19:33:13.678868 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":4}
I20260926 19:33:13.688512 8913 raft_consensus.cc:1261] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:13.692960 9294 consensus_queue.cc:1035] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:13.700400 9056 raft_consensus.cc:1261] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:13.702127 9320 consensus_queue.cc:1035] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:13.703110 9291 mvcc.cc:205] Tried to move back new op lower bound from 7333688089341161472 to 7333688089135017984. Current Snapshot: MvccSnapshot[applied={T|T < 7333688089341161472}]
W20260926 19:33:13.729959 9288 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 19:33:15.349072 8754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:15.349345 8754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:15.353475 8754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Sending DeleteTablet for 3 replicas of tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0
I20260926 19:33:15.354347 8893 tablet_service.cc:1514] Processing DeleteTablet for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:15 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33448
I20260926 19:33:15.354457 9032 tablet_service.cc:1514] Processing DeleteTablet for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:15 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43694
I20260926 19:33:15.354775 9171 tablet_service.cc:1514] Processing DeleteTablet for tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:15 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33886
I20260926 19:33:15.355048 9345 tablet_replica.cc:330] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: stopping tablet replica
I20260926 19:33:15.355283 9345 raft_consensus.cc:2229] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:15.355654 9345 raft_consensus.cc:2258] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:15.362054 9345 ts_tablet_manager.cc:1905] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:15.363668 9347 tablet_replica.cc:330] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: stopping tablet replica
I20260926 19:33:15.363946 9347 raft_consensus.cc:2229] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:15.364822 9347 raft_consensus.cc:2258] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:15.365595 9346 tablet_replica.cc:330] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: stopping tablet replica
I20260926 19:33:15.365883 9346 raft_consensus.cc:2229] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:15.366137 9347 ts_tablet_manager.cc:1905] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:15.366181 9346 raft_consensus.cc:2258] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:15.367321 9346 ts_tablet_manager.cc:1905] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:15.370975 9345 ts_tablet_manager.cc:1918] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 19:33:15.371206 9345 log.cc:1196] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/86ed5c1b53dd417cbbcb0d6e8e27c5f0
I20260926 19:33:15.371759 9345 ts_tablet_manager.cc:1939] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P b79eed4702d648bfa8dee79ecee570e1: Deleting consensus metadata
I20260926 19:33:15.373034 8743 catalog_manager.cc:4841] TS b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561): tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 (table test-table [id=18a418c40c0544fab760521221d4d89c]) successfully deleted
I20260926 19:33:15.373548 9347 ts_tablet_manager.cc:1918] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 19:33:15.373719 9347 log.cc:1196] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/86ed5c1b53dd417cbbcb0d6e8e27c5f0
I20260926 19:33:15.374313 9347 ts_tablet_manager.cc:1939] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P dfa36727c8964c7eaea513e4a0601ec5: Deleting consensus metadata
I20260926 19:33:15.374595 9346 ts_tablet_manager.cc:1918] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 19:33:15.374756 9346 log.cc:1196] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/86ed5c1b53dd417cbbcb0d6e8e27c5f0
I20260926 19:33:15.375200 9346 ts_tablet_manager.cc:1939] T 86ed5c1b53dd417cbbcb0d6e8e27c5f0 P e10440fe5ae14ab5a33921e559492c3e: Deleting consensus metadata
I20260926 19:33:15.375593 8742 catalog_manager.cc:4841] TS dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195:38147): tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 (table test-table [id=18a418c40c0544fab760521221d4d89c]) successfully deleted
I20260926 19:33:15.376430 8741 catalog_manager.cc:4841] TS e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059): tablet 86ed5c1b53dd417cbbcb0d6e8e27c5f0 (table test-table [id=18a418c40c0544fab760521221d4d89c]) successfully deleted
Sep 26 19:33:15 dist-test-slave-phbh krb5kdc[8711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 19:33:15 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:15.412233 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:15.395621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.6.215.254:36299)
0926 19:33:15.396213 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:15.396221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:15.396249 (+ 28us) server_negotiation.cc:407] Connection header received
0926 19:33:15.396483 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:15.396487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:15.396548 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:15.396660 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:15.397980 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:15.398737 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:15.400007 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:15.400422 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:15.400552 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:15.404656 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:15.404661 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:15.404664 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:15.404696 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:15.408232 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:15.409175 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:15.409180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:15.409184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:15.409371 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:15.409844 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:15.409865 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:15.409869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:15.410285 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:15.410592 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:15.411578 (+ 986us) server_negotiation.cc:299] Negotiation successful
0926 19:33:15.411847 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":266,"threads_started":1}
I20260926 19:33:15.416416 8754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:15.417109 8754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:15.423734 8893 tablet_service.cc:1467] Processing CreateTablet for tablet 542ba0f63b6b4aa484c37e9ed424e286 (DEFAULT_TABLE table=test-table [id=8122eb7e7302447a83153019b2d52420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:15.423899 9032 tablet_service.cc:1467] Processing CreateTablet for tablet 542ba0f63b6b4aa484c37e9ed424e286 (DEFAULT_TABLE table=test-table [id=8122eb7e7302447a83153019b2d52420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:15.424362 8893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542ba0f63b6b4aa484c37e9ed424e286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:15.424379 9032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542ba0f63b6b4aa484c37e9ed424e286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:15.429929 9356 tablet_bootstrap.cc:492] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: Bootstrap starting.
I20260926 19:33:15.430469 9357 tablet_bootstrap.cc:492] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: Bootstrap starting.
I20260926 19:33:15.430487 9171 tablet_service.cc:1467] Processing CreateTablet for tablet 542ba0f63b6b4aa484c37e9ed424e286 (DEFAULT_TABLE table=test-table [id=8122eb7e7302447a83153019b2d52420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:15.430943 9171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 542ba0f63b6b4aa484c37e9ed424e286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:15.432161 9357 tablet_bootstrap.cc:654] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:15.432309 9356 tablet_bootstrap.cc:654] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:15.439999 9357 tablet_bootstrap.cc:492] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: No bootstrap required, opened a new log
I20260926 19:33:15.440269 9357 ts_tablet_manager.cc:1397] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.005s
I20260926 19:33:15.440408 9360 tablet_bootstrap.cc:492] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: Bootstrap starting.
I20260926 19:33:15.440609 9356 tablet_bootstrap.cc:492] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: No bootstrap required, opened a new log
I20260926 19:33:15.440799 9356 ts_tablet_manager.cc:1397] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260926 19:33:15.441542 9356 raft_consensus.cc:348] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.441807 9356 raft_consensus.cc:374] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:15.441231 9357 raft_consensus.cc:348] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.441912 9356 raft_consensus.cc:729] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10440fe5ae14ab5a33921e559492c3e, State: Initialized, Role: FOLLOWER
I20260926 19:33:15.441964 9357 raft_consensus.cc:374] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:15.442062 9357 raft_consensus.cc:729] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79eed4702d648bfa8dee79ecee570e1, State: Initialized, Role: FOLLOWER
I20260926 19:33:15.442226 9356 consensus_queue.cc:260] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.442349 9357 consensus_queue.cc:260] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.442641 9360 tablet_bootstrap.cc:654] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:15.444725 9357 ts_tablet_manager.cc:1428] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 19:33:15.445263 9356 ts_tablet_manager.cc:1428] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260926 19:33:15.446553 9360 tablet_bootstrap.cc:492] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: No bootstrap required, opened a new log
I20260926 19:33:15.446708 9360 ts_tablet_manager.cc:1397] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 19:33:15.447511 9360 raft_consensus.cc:348] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.447741 9360 raft_consensus.cc:374] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:15.447837 9360 raft_consensus.cc:729] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Initialized, Role: FOLLOWER
I20260926 19:33:15.448096 9360 consensus_queue.cc:260] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.448623 9360 ts_tablet_manager.cc:1428] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 19:33:15.579984 9325 raft_consensus.cc:482] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:15.580169 9325 raft_consensus.cc:504] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.580972 9325 leader_election.cc:288] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:15.581501 8912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542ba0f63b6b4aa484c37e9ed424e286" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1" is_pre_election: true
I20260926 19:33:15.581570 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542ba0f63b6b4aa484c37e9ed424e286" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e" is_pre_election: true
I20260926 19:33:15.581756 8912 raft_consensus.cc:2454] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:15.581851 9056 raft_consensus.cc:2454] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:15.582274 9125 leader_election.cc:302] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b79eed4702d648bfa8dee79ecee570e1, dfa36727c8964c7eaea513e4a0601ec5; no voters:
I20260926 19:33:15.582540 9325 raft_consensus.cc:2781] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:15.582690 9325 raft_consensus.cc:482] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:15.582814 9325 raft_consensus.cc:3037] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:15.584954 9325 raft_consensus.cc:504] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.585475 9325 leader_election.cc:288] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Requested vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:15.586048 8912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542ba0f63b6b4aa484c37e9ed424e286" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1"
I20260926 19:33:15.586048 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "542ba0f63b6b4aa484c37e9ed424e286" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e"
I20260926 19:33:15.586262 9056 raft_consensus.cc:3037] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:15.586263 8912 raft_consensus.cc:3037] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:15.589728 8912 raft_consensus.cc:2454] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:15.589723 9056 raft_consensus.cc:2454] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:15.590204 9123 leader_election.cc:302] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfa36727c8964c7eaea513e4a0601ec5, e10440fe5ae14ab5a33921e559492c3e; no voters:
I20260926 19:33:15.590513 9325 raft_consensus.cc:2781] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:15.590631 9325 raft_consensus.cc:686] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 LEADER]: Becoming Leader. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Running, Role: LEADER
I20260926 19:33:15.590891 9325 consensus_queue.cc:237] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.593597 8754 catalog_manager.cc:5495] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } health_report { overall_health: HEALTHY } } }
I20260926 19:33:15.634958 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:15.629906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.6.215.195:38147)
0926 19:33:15.630444 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:15.630448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:15.630465 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:33:15.630512 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:15.630516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:15.630612 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:15.630728 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:15.631108 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:15.631867 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:15.633246 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:15.633667 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:15.633801 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:15.634230 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:15.634337 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:15.634438 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:15.634577 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 19:33:15.634647 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":272,"threads_started":1}
I20260926 19:33:15.643615 9056 raft_consensus.cc:1261] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:15.643877 8912 raft_consensus.cc:1261] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:15.644167 9325 consensus_queue.cc:1035] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:15.644536 9325 consensus_queue.cc:1035] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:15.685130 8756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:15.685359 8756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:15.688141 8756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Sending DeleteTablet for 3 replicas of tablet 542ba0f63b6b4aa484c37e9ed424e286
I20260926 19:33:15.688930 8893 tablet_service.cc:1514] Processing DeleteTablet for tablet 542ba0f63b6b4aa484c37e9ed424e286 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:15 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33448
I20260926 19:33:15.689150 9170 tablet_service.cc:1514] Processing DeleteTablet for tablet 542ba0f63b6b4aa484c37e9ed424e286 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:15 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33886
I20260926 19:33:15.689224 9345 tablet_replica.cc:330] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: stopping tablet replica
I20260926 19:33:15.689411 9345 raft_consensus.cc:2229] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:15.689430 9347 tablet_replica.cc:330] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: stopping tablet replica
I20260926 19:33:15.689575 9032 tablet_service.cc:1514] Processing DeleteTablet for tablet 542ba0f63b6b4aa484c37e9ed424e286 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:15 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43694
I20260926 19:33:15.689603 9345 raft_consensus.cc:2258] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:15.689627 9347 raft_consensus.cc:2229] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:15.689831 9346 tablet_replica.cc:330] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: stopping tablet replica
I20260926 19:33:15.689955 9347 raft_consensus.cc:2258] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:15.690021 9346 raft_consensus.cc:2229] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:15.690188 9346 raft_consensus.cc:2258] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:15.690761 9345 ts_tablet_manager.cc:1905] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:15.690604 8756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 19:33:15.691090 9347 ts_tablet_manager.cc:1905] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 19:33:15.691250 8756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:15.693156 9346 ts_tablet_manager.cc:1905] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:15.696595 9345 ts_tablet_manager.cc:1918] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:15.696740 9345 log.cc:1196] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/542ba0f63b6b4aa484c37e9ed424e286
I20260926 19:33:15.697154 9345 ts_tablet_manager.cc:1939] T 542ba0f63b6b4aa484c37e9ed424e286 P b79eed4702d648bfa8dee79ecee570e1: Deleting consensus metadata
I20260926 19:33:15.698261 8893 tablet_service.cc:1467] Processing CreateTablet for tablet c1466fce95a54ea7bb19f025badf4fd2 (DEFAULT_TABLE table=test-table [id=0b75a17c05104efc8faff1f33e40f4dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:15.698513 8743 catalog_manager.cc:4841] TS b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561): tablet 542ba0f63b6b4aa484c37e9ed424e286 (table test-table [id=8122eb7e7302447a83153019b2d52420]) successfully deleted
I20260926 19:33:15.698537 9170 tablet_service.cc:1467] Processing CreateTablet for tablet c1466fce95a54ea7bb19f025badf4fd2 (DEFAULT_TABLE table=test-table [id=0b75a17c05104efc8faff1f33e40f4dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:15.698727 8893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1466fce95a54ea7bb19f025badf4fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:15.699332 9170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1466fce95a54ea7bb19f025badf4fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:15.700071 9346 ts_tablet_manager.cc:1918] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:15.700201 9346 log.cc:1196] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/542ba0f63b6b4aa484c37e9ed424e286
I20260926 19:33:15.700598 9346 ts_tablet_manager.cc:1939] T 542ba0f63b6b4aa484c37e9ed424e286 P e10440fe5ae14ab5a33921e559492c3e: Deleting consensus metadata
I20260926 19:33:15.701659 8741 catalog_manager.cc:4841] TS e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059): tablet 542ba0f63b6b4aa484c37e9ed424e286 (table test-table [id=8122eb7e7302447a83153019b2d52420]) successfully deleted
I20260926 19:33:15.701778 9032 tablet_service.cc:1467] Processing CreateTablet for tablet c1466fce95a54ea7bb19f025badf4fd2 (DEFAULT_TABLE table=test-table [id=0b75a17c05104efc8faff1f33e40f4dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:15.702565 9032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1466fce95a54ea7bb19f025badf4fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:15.703408 9357 tablet_bootstrap.cc:492] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: Bootstrap starting.
I20260926 19:33:15.704958 9357 tablet_bootstrap.cc:654] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:15.706157 9360 tablet_bootstrap.cc:492] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: Bootstrap starting.
I20260926 19:33:15.707707 9360 tablet_bootstrap.cc:654] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:15.708985 9356 tablet_bootstrap.cc:492] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: Bootstrap starting.
I20260926 19:33:15.711328 9347 ts_tablet_manager.cc:1918] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:33:15.711542 9347 log.cc:1196] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/542ba0f63b6b4aa484c37e9ed424e286
I20260926 19:33:15.711864 9356 tablet_bootstrap.cc:654] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:15.712183 9347 ts_tablet_manager.cc:1939] T 542ba0f63b6b4aa484c37e9ed424e286 P dfa36727c8964c7eaea513e4a0601ec5: Deleting consensus metadata
I20260926 19:33:15.713567 8742 catalog_manager.cc:4841] TS dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195:38147): tablet 542ba0f63b6b4aa484c37e9ed424e286 (table test-table [id=8122eb7e7302447a83153019b2d52420]) successfully deleted
I20260926 19:33:15.714573 9357 tablet_bootstrap.cc:492] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: No bootstrap required, opened a new log
I20260926 19:33:15.714767 9357 ts_tablet_manager.cc:1397] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260926 19:33:15.715199 9360 tablet_bootstrap.cc:492] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: No bootstrap required, opened a new log
I20260926 19:33:15.715212 9356 tablet_bootstrap.cc:492] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: No bootstrap required, opened a new log
I20260926 19:33:15.715364 9360 ts_tablet_manager.cc:1397] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 19:33:15.715364 9356 ts_tablet_manager.cc:1397] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260926 19:33:15.715514 9357 raft_consensus.cc:348] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.715843 9357 raft_consensus.cc:374] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:15.715976 9357 raft_consensus.cc:729] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79eed4702d648bfa8dee79ecee570e1, State: Initialized, Role: FOLLOWER
I20260926 19:33:15.716046 9360 raft_consensus.cc:348] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.716046 9356 raft_consensus.cc:348] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.716295 9356 raft_consensus.cc:374] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:15.716295 9360 raft_consensus.cc:374] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:15.716377 9360 raft_consensus.cc:729] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Initialized, Role: FOLLOWER
I20260926 19:33:15.716420 9356 raft_consensus.cc:729] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10440fe5ae14ab5a33921e559492c3e, State: Initialized, Role: FOLLOWER
I20260926 19:33:15.716638 9356 consensus_queue.cc:260] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.716638 9360 consensus_queue.cc:260] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.717182 9360 ts_tablet_manager.cc:1428] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 19:33:15.717199 9356 ts_tablet_manager.cc:1428] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 19:33:15.717674 9357 consensus_queue.cc:260] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.718418 9357 ts_tablet_manager.cc:1428] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260926 19:33:15.744671 9328 raft_consensus.cc:482] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:15.744961 9328 raft_consensus.cc:504] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.745769 9328 leader_election.cc:288] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:15.746186 8912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1466fce95a54ea7bb19f025badf4fd2" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1" is_pre_election: true
I20260926 19:33:15.746289 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1466fce95a54ea7bb19f025badf4fd2" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e" is_pre_election: true
I20260926 19:33:15.746490 8912 raft_consensus.cc:2454] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:15.746500 9056 raft_consensus.cc:2454] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 0.
I20260926 19:33:15.746937 9123 leader_election.cc:302] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b79eed4702d648bfa8dee79ecee570e1, dfa36727c8964c7eaea513e4a0601ec5, e10440fe5ae14ab5a33921e559492c3e; no voters:
I20260926 19:33:15.747205 9328 raft_consensus.cc:2781] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:15.747308 9328 raft_consensus.cc:482] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:15.747421 9328 raft_consensus.cc:3037] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:15.749477 9328 raft_consensus.cc:504] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.749976 9328 leader_election.cc:288] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Requested vote from peers b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561), e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059)
I20260926 19:33:15.750486 9056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1466fce95a54ea7bb19f025badf4fd2" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10440fe5ae14ab5a33921e559492c3e"
I20260926 19:33:15.750500 8912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1466fce95a54ea7bb19f025badf4fd2" candidate_uuid: "dfa36727c8964c7eaea513e4a0601ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b79eed4702d648bfa8dee79ecee570e1"
I20260926 19:33:15.750694 8912 raft_consensus.cc:3037] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:15.750694 9056 raft_consensus.cc:3037] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:15.753638 9056 raft_consensus.cc:2454] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:15.753631 8912 raft_consensus.cc:2454] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa36727c8964c7eaea513e4a0601ec5 in term 1.
I20260926 19:33:15.754019 9123 leader_election.cc:302] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfa36727c8964c7eaea513e4a0601ec5, e10440fe5ae14ab5a33921e559492c3e; no voters:
I20260926 19:33:15.754369 9328 raft_consensus.cc:2781] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:15.754561 9328 raft_consensus.cc:686] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 LEADER]: Becoming Leader. State: Replica: dfa36727c8964c7eaea513e4a0601ec5, State: Running, Role: LEADER
I20260926 19:33:15.755089 9328 consensus_queue.cc:237] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } }
I20260926 19:33:15.757968 8756 catalog_manager.cc:5495] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "dfa36727c8964c7eaea513e4a0601ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfa36727c8964c7eaea513e4a0601ec5" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 38147 } health_report { overall_health: HEALTHY } } }
I20260926 19:33:15.784278 9055 raft_consensus.cc:1261] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:15.784307 8912 raft_consensus.cc:1261] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Refusing update from remote peer dfa36727c8964c7eaea513e4a0601ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:15.784834 9325 consensus_queue.cc:1035] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79eed4702d648bfa8dee79ecee570e1" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:15.785091 9328 consensus_queue.cc:1035] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e10440fe5ae14ab5a33921e559492c3e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 35059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:33:15.803397 9326 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 19:33:17.387454 8754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:17.387718 8754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:17.391445 8754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e727b0c8f383404f9c08b2e202c3241f: Sending DeleteTablet for 3 replicas of tablet c1466fce95a54ea7bb19f025badf4fd2
I20260926 19:33:17.392261 8893 tablet_service.cc:1514] Processing DeleteTablet for tablet c1466fce95a54ea7bb19f025badf4fd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:17 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33448
I20260926 19:33:17.392316 9032 tablet_service.cc:1514] Processing DeleteTablet for tablet c1466fce95a54ea7bb19f025badf4fd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:17 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43694
I20260926 19:33:17.392418 9171 tablet_service.cc:1514] Processing DeleteTablet for tablet c1466fce95a54ea7bb19f025badf4fd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:17 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33886
I20260926 19:33:17.393011 9397 tablet_replica.cc:330] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: stopping tablet replica
I20260926 19:33:17.393146 9399 tablet_replica.cc:330] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: stopping tablet replica
I20260926 19:33:17.393338 9399 raft_consensus.cc:2229] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:17.393401 9397 raft_consensus.cc:2229] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:17.393592 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8827
I20260926 19:33:17.393705 9399 raft_consensus.cc:2258] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:17.393921 9397 raft_consensus.cc:2258] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:17.394259 9398 tablet_replica.cc:330] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: stopping tablet replica
I20260926 19:33:17.394498 9398 raft_consensus.cc:2229] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:17.394773 9398 raft_consensus.cc:2258] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:17.395308 9397 ts_tablet_manager.cc:1905] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:17.396009 9399 ts_tablet_manager.cc:1905] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:17.396723 9398 ts_tablet_manager.cc:1905] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:17.402917 9399 ts_tablet_manager.cc:1918] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 19:33:17.403116 9399 log.cc:1196] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/c1466fce95a54ea7bb19f025badf4fd2
I20260926 19:33:17.403517 9398 ts_tablet_manager.cc:1918] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 19:33:17.403705 9398 log.cc:1196] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/c1466fce95a54ea7bb19f025badf4fd2
I20260926 19:33:17.403694 9399 ts_tablet_manager.cc:1939] T c1466fce95a54ea7bb19f025badf4fd2 P dfa36727c8964c7eaea513e4a0601ec5: Deleting consensus metadata
I20260926 19:33:17.403962 9397 ts_tablet_manager.cc:1918] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 19:33:17.404156 9397 log.cc:1196] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/c1466fce95a54ea7bb19f025badf4fd2
I20260926 19:33:17.404166 9398 ts_tablet_manager.cc:1939] T c1466fce95a54ea7bb19f025badf4fd2 P e10440fe5ae14ab5a33921e559492c3e: Deleting consensus metadata
I20260926 19:33:17.404721 9397 ts_tablet_manager.cc:1939] T c1466fce95a54ea7bb19f025badf4fd2 P b79eed4702d648bfa8dee79ecee570e1: Deleting consensus metadata
I20260926 19:33:17.405129 8742 catalog_manager.cc:4841] TS dfa36727c8964c7eaea513e4a0601ec5 (127.6.215.195:38147): tablet c1466fce95a54ea7bb19f025badf4fd2 (table test-table [id=0b75a17c05104efc8faff1f33e40f4dd]) successfully deleted
I20260926 19:33:17.405129 8741 catalog_manager.cc:4841] TS e10440fe5ae14ab5a33921e559492c3e (127.6.215.194:35059): tablet c1466fce95a54ea7bb19f025badf4fd2 (table test-table [id=0b75a17c05104efc8faff1f33e40f4dd]) successfully deleted
I20260926 19:33:17.406018 8743 catalog_manager.cc:4841] TS b79eed4702d648bfa8dee79ecee570e1 (127.6.215.193:46561): tablet c1466fce95a54ea7bb19f025badf4fd2 (table test-table [id=0b75a17c05104efc8faff1f33e40f4dd]) successfully deleted
Sep 26 19:33:17 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:17.427376 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:17.395001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44456 (local address 127.6.215.193:46561)
0926 19:33:17.405894 (+ 10893us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:17.405901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:17.405961 (+ 60us) server_negotiation.cc:407] Connection header received
0926 19:33:17.406192 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:17.406197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:17.406274 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:17.406392 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:17.407644 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:17.408383 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:17.409662 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:17.410074 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:17.410203 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:17.420915 (+ 10712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:17.420931 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:17.420948 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:17.420987 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:17.424373 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:17.425150 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:17.425158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:17.425160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:17.425318 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:17.425823 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:17.425826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:17.425828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:17.426227 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:17.426437 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:17.426747 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 19:33:17.427000 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10756,"thread_start_us":10587,"threads_started":1}
W20260926 19:33:17.428135 8958 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44456
W20260926 19:33:17.428910 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8827: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:17.429159 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 8827
I20260926 19:33:17.447289 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8967
Sep 26 19:33:17 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:33:17.461954 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:17.448051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55548 (local address 127.6.215.194:35059)
0926 19:33:17.448589 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:17.448593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:17.448608 (+ 15us) server_negotiation.cc:407] Connection header received
0926 19:33:17.448660 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:17.448664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:17.448721 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:17.448876 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:17.450128 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:17.450917 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:17.452127 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:17.452504 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:17.452599 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:17.456712 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:17.456725 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:17.456745 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:17.456847 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:17.459495 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:17.460136 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:17.460142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:17.460144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:17.460281 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:17.460574 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:17.460577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:17.460579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:17.460993 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:17.461199 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:17.461439 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 19:33:17.461594 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":326,"threads_started":1}
W20260926 19:33:17.462565 9097 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55548
W20260926 19:33:17.463128 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8967: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:17.463260 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 8967
I20260926 19:33:17.485376 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9106
Sep 26 19:33:17 dist-test-slave-phbh krb5kdc[8711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:17.500651 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:17.486291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.6.215.195:38147)
0926 19:33:17.486799 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:17.486803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:17.486819 (+ 16us) server_negotiation.cc:407] Connection header received
0926 19:33:17.486872 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:17.486876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:17.486932 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:17.487025 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:17.488278 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:17.489105 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:17.490262 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:17.490628 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:17.490716 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:17.494843 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:17.494857 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:17.494872 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:17.494904 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:17.497934 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:17.498624 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:17.498642 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:17.498646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:17.498774 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:17.499143 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:17.499153 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:17.499155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:17.499490 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:17.499793 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:17.500076 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 19:33:17.500318 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":265,"threads_started":1}
W20260926 19:33:17.501386 9236 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38394
W20260926 19:33:17.502038 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9106: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:17.502203 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 9106
I20260926 19:33:17.527418 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 8724
I20260926 19:33:17.528570 8796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:17.643343 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 8724
2026-09-26T19:33:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8272 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:17 dist-test-slave-phbh krb5kdc[9407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:17 dist-test-slave-phbh krb5kdc[9407](info): set up 2 sockets
Sep 26 19:33:17 dist-test-slave-phbh krb5kdc[9407](info): commencing operation
krb5kdc: starting...
W20260926 19:33:18.753769 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:18 dist-test-slave-phbh krb5kdc[9407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:18.945056 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46451
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:46451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:19.262969 9420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:19.263260 9420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:19.263350 9420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:19.270413 9420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:19.270520 9420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:19.270565 9420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:19.270617 9420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:19.270660 9420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:19.280315 9420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:46451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:19.281561 9420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:19.282927 9420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:19.295084 9425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:19.295130 9428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:19.295104 9426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:19.295904 9420 server_base.cc:1034] running on GCE node
I20260926 19:33:19.296712 9420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:19.298133 9420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:19.299366 9420 hybrid_clock.cc:648] HybridClock initialized: now 1790451199299306 us; error 47 us; skew 500 ppm
I20260926 19:33:19.299701 9420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:19 dist-test-slave-phbh krb5kdc[9407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451199, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:19.302937 9420 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:19.304349 9420 webserver.cc:460] Webserver started at http://127.6.215.254:34807/ using document root <none> and password file <none>
I20260926 19:33:19.305099 9420 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:19.305233 9420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:19.305544 9420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:19.308445 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "4a8cb67c6848452683aed2ee6656fe5a"
format_stamp: "Formatted at 2026-09-26 19:33:19 on dist-test-slave-phbh"
server_key: "43af648da7dcf9a6279c1ed72c25a209"
server_key_iv: "44365a8fd2b340039a24cd0df62e6d8f"
server_key_version: "encryptionkey@0"
I20260926 19:33:19.309238 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "4a8cb67c6848452683aed2ee6656fe5a"
format_stamp: "Formatted at 2026-09-26 19:33:19 on dist-test-slave-phbh"
server_key: "43af648da7dcf9a6279c1ed72c25a209"
server_key_iv: "44365a8fd2b340039a24cd0df62e6d8f"
server_key_version: "encryptionkey@0"
I20260926 19:33:19.314085 9420 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 19:33:19.317366 9435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:19.318600 9420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:33:19.318776 9420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "4a8cb67c6848452683aed2ee6656fe5a"
format_stamp: "Formatted at 2026-09-26 19:33:19 on dist-test-slave-phbh"
server_key: "43af648da7dcf9a6279c1ed72c25a209"
server_key_iv: "44365a8fd2b340039a24cd0df62e6d8f"
server_key_version: "encryptionkey@0"
I20260926 19:33:19.318964 9420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:19.340512 9420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:19.345023 9420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:19.345314 9420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:19.362995 9420 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:46451
I20260926 19:33:19.363016 9486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:46451 every 8 connection(s)
I20260926 19:33:19.364385 9420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:19.368016 9487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:19.373972 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 9420
I20260926 19:33:19.374270 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:19.374881 7007 external_mini_cluster.cc:1428] Setting key 69854ea78df6d38c0db634fd060f8823
I20260926 19:33:19.378842 9487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a: Bootstrap starting.
I20260926 19:33:19.381910 9487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:19.382946 9487 log.cc:824] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:19.385720 9487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a: No bootstrap required, opened a new log
Sep 26 19:33:19 dist-test-slave-phbh krb5kdc[9407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:19.393508 9487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8cb67c6848452683aed2ee6656fe5a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46451 } }
I20260926 19:33:19.393914 9487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:19.394030 9487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a8cb67c6848452683aed2ee6656fe5a, State: Initialized, Role: FOLLOWER
I20260926 19:33:19.394748 9487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8cb67c6848452683aed2ee6656fe5a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46451 } }
I20260926 19:33:19.395051 9487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:19.395238 9487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:19.395435 9487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:19.397042 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:19.376928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.6.215.254:46451)
0926 19:33:19.377752 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:19.377765 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:19.377806 (+ 41us) server_negotiation.cc:407] Connection header received
0926 19:33:19.379528 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:19.379549 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:19.379929 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:19.380327 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:19.381814 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:19.383278 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:19.384473 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:19.385289 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:19.385478 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:19.389551 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:19.389560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:19.389578 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:19.389624 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:19.392684 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:19.393375 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:19.393382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:19.393388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:19.393545 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:19.393963 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:19.393969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:19.393973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:19.394677 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:19.394989 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:19.395411 (+ 422us) server_negotiation.cc:299] Negotiation successful
0926 19:33:19.395862 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":402,"threads_started":1}
I20260926 19:33:19.398269 9487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8cb67c6848452683aed2ee6656fe5a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46451 } }
I20260926 19:33:19.398833 9487 leader_election.cc:302] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a8cb67c6848452683aed2ee6656fe5a; no voters:
I20260926 19:33:19.399498 9487 leader_election.cc:288] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:19.399611 9492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:19.400745 9492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [term 1 LEADER]: Becoming Leader. State: Replica: 4a8cb67c6848452683aed2ee6656fe5a, State: Running, Role: LEADER
I20260926 19:33:19.401333 9492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8cb67c6848452683aed2ee6656fe5a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46451 } }
I20260926 19:33:19.401799 9487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:19.403975 9494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a8cb67c6848452683aed2ee6656fe5a. Latest consensus state: current_term: 1 leader_uuid: "4a8cb67c6848452683aed2ee6656fe5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8cb67c6848452683aed2ee6656fe5a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46451 } } }
I20260926 19:33:19.404315 9494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:19.404830 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a8cb67c6848452683aed2ee6656fe5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a8cb67c6848452683aed2ee6656fe5a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46451 } } }
I20260926 19:33:19.405118 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:19.406328 9497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:19.412451 9497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:19.422314 9497 catalog_manager.cc:1347] Generated new cluster ID: 8e8912fe259b45709fcb5becfbc6ad43
I20260926 19:33:19.422463 9497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:19.450618 9497 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:19.451972 9497 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:19.460003 9497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a8cb67c6848452683aed2ee6656fe5a: Generated new TSK 0
I20260926 19:33:19.460894 9497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:19.513238 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46451
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:19.843755 9513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:19.844039 9513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:19.844214 9513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:19.851397 9513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:19.851507 9513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:19.851711 9513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:19.861294 9513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:19.862597 9513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:19.863911 9513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:19.877633 9518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:19.877595 9521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:19.877681 9519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:19.882449 9513 server_base.cc:1034] running on GCE node
I20260926 19:33:19.883090 9513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:19.884058 9513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:19.885267 9513 hybrid_clock.cc:648] HybridClock initialized: now 1790451199885210 us; error 43 us; skew 500 ppm
I20260926 19:33:19.885555 9513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:19 dist-test-slave-phbh krb5kdc[9407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451199, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:19.888706 9513 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:19.890197 9513 webserver.cc:460] Webserver started at http://127.6.215.193:34123/ using document root <none> and password file <none>
I20260926 19:33:19.890913 9513 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:19.891055 9513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:19.891356 9513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:19.894284 9513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "f6846f5d73d94eb1873a8197f0fc4dc7"
format_stamp: "Formatted at 2026-09-26 19:33:19 on dist-test-slave-phbh"
server_key: "8be646bc50c1ba2eb721a7adf5b7c5fe"
server_key_iv: "5166769abd6882da888ea77ab84740ff"
server_key_version: "encryptionkey@0"
I20260926 19:33:19.895048 9513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "f6846f5d73d94eb1873a8197f0fc4dc7"
format_stamp: "Formatted at 2026-09-26 19:33:19 on dist-test-slave-phbh"
server_key: "8be646bc50c1ba2eb721a7adf5b7c5fe"
server_key_iv: "5166769abd6882da888ea77ab84740ff"
server_key_version: "encryptionkey@0"
I20260926 19:33:19.900189 9513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 19:33:19.904153 9528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:19.905866 9513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:19.906032 9513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "f6846f5d73d94eb1873a8197f0fc4dc7"
format_stamp: "Formatted at 2026-09-26 19:33:19 on dist-test-slave-phbh"
server_key: "8be646bc50c1ba2eb721a7adf5b7c5fe"
server_key_iv: "5166769abd6882da888ea77ab84740ff"
server_key_version: "encryptionkey@0"
I20260926 19:33:19.906273 9513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:19.930444 9513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:19.934921 9513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:19.935225 9513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:19.936177 9513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:19.937516 9513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:19.937621 9513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:19.937775 9513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:19.937848 9513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:19.970938 9513 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:40551
I20260926 19:33:19.970996 9640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:40551 every 8 connection(s)
I20260926 19:33:19.972247 9513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:19.973544 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 9513
I20260926 19:33:19.973774 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:19.974555 7007 external_mini_cluster.cc:1428] Setting key a1cc6c967aeb90049d0b8d87df9defd4
Sep 26 19:33:19 dist-test-slave-phbh krb5kdc[9407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451199, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:33:19.994200 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:19.975187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:42319 (local address 127.6.215.254:46451)
0926 19:33:19.975668 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:19.975675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:19.976817 (+ 1142us) server_negotiation.cc:407] Connection header received
0926 19:33:19.978620 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:19.978626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:19.978734 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:19.978852 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:19.980350 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:19.981112 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:19.983450 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:19.983855 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:19.983960 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:19.988294 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:19.988300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:19.988303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:19.988336 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:19.991030 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:19.991775 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:19.991788 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:19.991791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:19.991900 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:19.992350 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:19.992356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:19.992359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:19.992628 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:19.992817 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:19.993616 (+ 799us) server_negotiation.cc:299] Negotiation successful
0926 19:33:19.993832 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":248,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
I20260926 19:33:19.995347 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:19.975797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46451 (local address 127.6.215.193:42319)
0926 19:33:19.976627 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:19.976688 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:19.978358 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:19.979086 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:19.979094 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:19.979613 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:19.980178 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:19.980195 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:19.981274 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:19.981279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:19.983282 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:19.983300 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:19.984090 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:19.984100 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:19.984433 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:19.985217 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:19.985251 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:19.987994 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:19.991183 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:19.991192 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:19.991210 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:19.991640 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:19.992010 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:19.992015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:19.992019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:19.992235 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:19.992793 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:19.992802 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:19.993229 (+ 427us) client_negotiation.cc:769] Sending connection context
0926 19:33:19.993509 (+ 280us) client_negotiation.cc:240] Negotiation successful
0926 19:33:19.994053 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":304,"threads_started":1}
I20260926 19:33:19.996985 9641 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46451
I20260926 19:33:19.997331 9641 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:19.998230 9641 heartbeater.cc:511] Master 127.6.215.254:46451 requested a full tablet report, sending...
I20260926 19:33:20.000427 9452 ts_manager.cc:194] Registered new tserver with Master: f6846f5d73d94eb1873a8197f0fc4dc7 (127.6.215.193:40551)
I20260926 19:33:20.001927 9452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:42319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:20.019457 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46451
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:20.331979 9647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:20.332257 9647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:20.332409 9647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:20.339720 9647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:20.339835 9647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:20.340025 9647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:20.349925 9647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:20.351224 9647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:20.352548 9647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:20.365722 9653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:20.365746 9652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:20.366400 9655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:20.366677 9647 server_base.cc:1034] running on GCE node
I20260926 19:33:20.367317 9647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:20.368295 9647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:20.369517 9647 hybrid_clock.cc:648] HybridClock initialized: now 1790451200369458 us; error 52 us; skew 500 ppm
I20260926 19:33:20.369848 9647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:20 dist-test-slave-phbh krb5kdc[9407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451200, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:20.373152 9647 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:20.374773 9647 webserver.cc:460] Webserver started at http://127.6.215.194:37191/ using document root <none> and password file <none>
I20260926 19:33:20.375464 9647 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:20.375607 9647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:20.375928 9647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:20.378888 9647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "705aa4192c8c4453aa9917562ebd2225"
format_stamp: "Formatted at 2026-09-26 19:33:20 on dist-test-slave-phbh"
server_key: "cdf2bf15766998b3d2ff5ac139710907"
server_key_iv: "75e5365d3b17592dfdc0e7517c693d24"
server_key_version: "encryptionkey@0"
I20260926 19:33:20.379673 9647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "705aa4192c8c4453aa9917562ebd2225"
format_stamp: "Formatted at 2026-09-26 19:33:20 on dist-test-slave-phbh"
server_key: "cdf2bf15766998b3d2ff5ac139710907"
server_key_iv: "75e5365d3b17592dfdc0e7517c693d24"
server_key_version: "encryptionkey@0"
I20260926 19:33:20.384711 9647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 19:33:20.388473 9662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:20.389787 9647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:20.389968 9647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "705aa4192c8c4453aa9917562ebd2225"
format_stamp: "Formatted at 2026-09-26 19:33:20 on dist-test-slave-phbh"
server_key: "cdf2bf15766998b3d2ff5ac139710907"
server_key_iv: "75e5365d3b17592dfdc0e7517c693d24"
server_key_version: "encryptionkey@0"
I20260926 19:33:20.390134 9647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:20.407230 9647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:20.411760 9647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:20.412086 9647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:20.413156 9647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:20.414670 9647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:20.414747 9647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:20.414939 9647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:20.414992 9647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:20.448715 9647 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:37421
I20260926 19:33:20.448758 9774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:37421 every 8 connection(s)
I20260926 19:33:20.450110 9647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:20.457504 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 9647
I20260926 19:33:20.457728 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:20.458321 7007 external_mini_cluster.cc:1428] Setting key e7d8953f5c43b299f8d570eb135b232d
Sep 26 19:33:20 dist-test-slave-phbh krb5kdc[9407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451200, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:20.471951 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:20.452809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:59769 (local address 127.6.215.254:46451)
0926 19:33:20.452993 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:20.452998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:20.454245 (+ 1247us) server_negotiation.cc:407] Connection header received
0926 19:33:20.456273 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:20.456280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:20.456378 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:20.456501 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:20.458655 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.459526 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.461443 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.462216 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.462375 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:20.466088 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:20.466093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:20.466097 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:20.466126 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:20.468689 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:20.469628 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:20.469632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:20.469635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:20.469752 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:20.470224 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:20.470227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:20.470229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:20.470486 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:20.470591 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:20.471472 (+ 881us) server_negotiation.cc:299] Negotiation successful
0926 19:33:20.471637 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:33:20.473603 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:20.453273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46451 (local address 127.6.215.194:59769)
0926 19:33:20.454032 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:20.454094 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:20.455982 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:20.456747 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:20.456756 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:20.457583 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:20.458454 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:20.458485 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.459675 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.459680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:20.461291 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:20.461304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.462449 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.462453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:20.462644 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:20.463353 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:20.463375 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:20.465847 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:20.468862 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:20.468873 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:20.468893 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:20.469511 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:20.469863 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:20.469871 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:20.469877 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:20.470114 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:20.470609 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:20.470620 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:20.471087 (+ 467us) client_negotiation.cc:769] Sending connection context
0926 19:33:20.471392 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 19:33:20.472010 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":290,"threads_started":1}
I20260926 19:33:20.475281 9775 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46451
I20260926 19:33:20.475593 9775 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:20.476325 9775 heartbeater.cc:511] Master 127.6.215.254:46451 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:33:20.477860 9452 ts_manager.cc:194] Registered new tserver with Master: 705aa4192c8c4453aa9917562ebd2225 (127.6.215.194:37421)
I20260926 19:33:20.478683 9452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:59769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:20.503683 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46451
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:20.818905 9780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:20.819257 9780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:20.819425 9780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:20.826467 9780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:20.826620 9780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:20.826808 9780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:20.836155 9780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:20.837441 9780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:20.838719 9780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:20.851818 9786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:20.851838 9785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:20.851784 9788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:20.852871 9780 server_base.cc:1034] running on GCE node
I20260926 19:33:20.853518 9780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:20.854452 9780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:20.855684 9780 hybrid_clock.cc:648] HybridClock initialized: now 1790451200855623 us; error 43 us; skew 500 ppm
I20260926 19:33:20.856037 9780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:20 dist-test-slave-phbh krb5kdc[9407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451200, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:20.859216 9780 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:20.860630 9780 webserver.cc:460] Webserver started at http://127.6.215.195:45657/ using document root <none> and password file <none>
I20260926 19:33:20.861362 9780 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:20.861505 9780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:20.861814 9780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:20.864708 9780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "621b71fee82a4e17acde8e256f0cf1a2"
format_stamp: "Formatted at 2026-09-26 19:33:20 on dist-test-slave-phbh"
server_key: "796b0130d0d14ae3169cb7f1bdd3c411"
server_key_iv: "3b61f1a8ca37c57d73cf6d95547df00a"
server_key_version: "encryptionkey@0"
I20260926 19:33:20.865545 9780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "621b71fee82a4e17acde8e256f0cf1a2"
format_stamp: "Formatted at 2026-09-26 19:33:20 on dist-test-slave-phbh"
server_key: "796b0130d0d14ae3169cb7f1bdd3c411"
server_key_iv: "3b61f1a8ca37c57d73cf6d95547df00a"
server_key_version: "encryptionkey@0"
I20260926 19:33:20.870585 9780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:20.874163 9795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:20.875644 9780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:33:20.875840 9780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "621b71fee82a4e17acde8e256f0cf1a2"
format_stamp: "Formatted at 2026-09-26 19:33:20 on dist-test-slave-phbh"
server_key: "796b0130d0d14ae3169cb7f1bdd3c411"
server_key_iv: "3b61f1a8ca37c57d73cf6d95547df00a"
server_key_version: "encryptionkey@0"
I20260926 19:33:20.876063 9780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:20.898523 9780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:20.902704 9780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:20.902989 9780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:20.903906 9780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:20.905375 9780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:20.905478 9780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:20.905639 9780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:20.905727 9780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:20.938774 9780 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:42697
I20260926 19:33:20.938864 9907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:42697 every 8 connection(s)
I20260926 19:33:20.940163 9780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:20.941746 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 9780
I20260926 19:33:20.941903 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:20.942272 7007 external_mini_cluster.cc:1428] Setting key 53412b1afafb60c93cb69ddb97f9ee3b
Sep 26 19:33:20 dist-test-slave-phbh krb5kdc[9407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451200, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:20.960364 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:20.944173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54199 (local address 127.6.215.254:46451)
0926 19:33:20.944340 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:20.944345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:20.945804 (+ 1459us) server_negotiation.cc:407] Connection header received
0926 19:33:20.947477 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:20.947480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:20.947568 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:20.947651 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:20.949138 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.949894 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.951345 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.951705 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.951795 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:20.955093 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:20.955098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:20.955101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:20.955129 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:20.957813 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:20.958472 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:20.958476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:20.958478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:20.958593 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:20.958956 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:20.958959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:20.958961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:20.959210 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:20.959323 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:20.959948 (+ 625us) server_negotiation.cc:299] Negotiation successful
0926 19:33:20.960098 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 19:33:20.961364 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:20.944852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46451 (local address 127.6.215.195:54199)
0926 19:33:20.945652 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:20.945694 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:20.947233 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:20.947857 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:20.947865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:20.948364 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:20.948990 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:20.949013 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.950027 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.950030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:20.951224 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:20.951235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:20.951916 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:20.951921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:20.952131 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:20.952860 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:20.952882 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:20.954915 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:20.957929 (+ 3014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:20.957935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:20.957947 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:20.958362 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:20.958682 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:20.958685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:20.958688 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:20.958861 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:20.959321 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:20.959327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:20.959623 (+ 296us) client_negotiation.cc:769] Sending connection context
0926 19:33:20.959850 (+ 227us) client_negotiation.cc:240] Negotiation successful
0926 19:33:20.960205 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":337,"threads_started":1}
I20260926 19:33:20.962697 9908 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46451
I20260926 19:33:20.962952 9908 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:20.963577 9908 heartbeater.cc:511] Master 127.6.215.254:46451 requested a full tablet report, sending...
I20260926 19:33:20.964915 9452 ts_manager.cc:194] Registered new tserver with Master: 621b71fee82a4e17acde8e256f0cf1a2 (127.6.215.195:42697)
I20260926 19:33:20.965677 9452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:54199
I20260926 19:33:20.969826 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 19:33:21.000190 9643 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:20.997619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.6.215.254:46451)
0926 19:33:20.997857 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:20.997864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:20.998284 (+ 420us) server_negotiation.cc:407] Connection header received
0926 19:33:20.998494 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:20.998503 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:20.998566 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:20.998676 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:20.999911 (+ 1235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33660: BlockingRecv error: recv got EOF from 127.0.0.1:33660 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:33:21.003046 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9513
I20260926 19:33:21.004890 9641 heartbeater.cc:503] Master 127.6.215.254:46451 was elected leader, sending a full tablet report...
W20260926 19:33:21.006055 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9513: Not authorized: Client connection negotiation failed: client connection to 127.6.215.193:40551: server requires authentication, but client does not have Kerberos credentials available
I20260926 19:33:21.006238 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 9513
W20260926 19:33:21.021440 9438 connection.cc:535] server connection from 127.6.215.193:42319 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:33:21.021739 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9647
W20260926 19:33:21.024253 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9647: Not authorized: Client connection negotiation failed: client connection to 127.6.215.194:37421: server requires authentication, but client does not have Kerberos credentials available
I20260926 19:33:21.024418 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 9647
I20260926 19:33:21.039407 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9780
W20260926 19:33:21.042090 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9780: Not authorized: Client connection negotiation failed: client connection to 127.6.215.195:42697: server requires authentication, but client does not have Kerberos credentials available
I20260926 19:33:21.042209 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 9780
I20260926 19:33:21.057492 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9420
I20260926 19:33:21.058696 9482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:21.160085 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 9420
2026-09-26T19:33:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3513 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:21 dist-test-slave-phbh krb5kdc[9925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:21 dist-test-slave-phbh krb5kdc[9925](info): set up 2 sockets
Sep 26 19:33:21 dist-test-slave-phbh krb5kdc[9925](info): commencing operation
krb5kdc: starting...
W20260926 19:33:22.237823 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.005s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:22 dist-test-slave-phbh krb5kdc[9925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:22.430092 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42371
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:22.747480 9938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:22.747820 9938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:22.747991 9938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:22.755245 9938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:22.755406 9938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:22.755515 9938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:22.755622 9938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:22.755718 9938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:22.765666 9938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:22.766991 9938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:22.768311 9938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:22.781172 9946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:22.781236 9944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:22.781199 9943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:22.781986 9938 server_base.cc:1034] running on GCE node
I20260926 19:33:22.782681 9938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:22.784034 9938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:22.785225 9938 hybrid_clock.cc:648] HybridClock initialized: now 1790451202785187 us; error 35 us; skew 500 ppm
I20260926 19:33:22.785542 9938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:22 dist-test-slave-phbh krb5kdc[9925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451202, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:22.795881 9938 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:22.797408 9938 webserver.cc:460] Webserver started at http://127.6.215.254:45007/ using document root <none> and password file <none>
I20260926 19:33:22.798076 9938 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:22.798180 9938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:22.798437 9938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:22.801368 9938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "b6297cf45d4249bf8d13582a20a0f8c2"
format_stamp: "Formatted at 2026-09-26 19:33:22 on dist-test-slave-phbh"
server_key: "ca161429fc5096da7de2d3ae0c81abed"
server_key_iv: "1d397d4e1927930a6dac852bade9f570"
server_key_version: "encryptionkey@0"
I20260926 19:33:22.802073 9938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "b6297cf45d4249bf8d13582a20a0f8c2"
format_stamp: "Formatted at 2026-09-26 19:33:22 on dist-test-slave-phbh"
server_key: "ca161429fc5096da7de2d3ae0c81abed"
server_key_iv: "1d397d4e1927930a6dac852bade9f570"
server_key_version: "encryptionkey@0"
I20260926 19:33:22.807024 9938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:22.810300 9953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:22.811587 9938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:33:22.811736 9938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "b6297cf45d4249bf8d13582a20a0f8c2"
format_stamp: "Formatted at 2026-09-26 19:33:22 on dist-test-slave-phbh"
server_key: "ca161429fc5096da7de2d3ae0c81abed"
server_key_iv: "1d397d4e1927930a6dac852bade9f570"
server_key_version: "encryptionkey@0"
I20260926 19:33:22.811887 9938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:22.845242 9938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:22.849262 9938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:22.849501 9938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:22.868001 9938 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42371
I20260926 19:33:22.868036 10004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42371 every 8 connection(s)
I20260926 19:33:22.869529 9938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:22.873406 10005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:22.879130 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 9938
I20260926 19:33:22.879351 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:22.880196 7007 external_mini_cluster.cc:1428] Setting key e03c3e03d67abcf057c8f98426ab81c7
I20260926 19:33:22.885536 10005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2: Bootstrap starting.
I20260926 19:33:22.888527 10005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:22.889608 10005 log.cc:824] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:22.892359 10005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2: No bootstrap required, opened a new log
Sep 26 19:33:22 dist-test-slave-phbh krb5kdc[9925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:22.901585 10005 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42371 } }
I20260926 19:33:22.902051 10005 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:22.902166 10005 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6297cf45d4249bf8d13582a20a0f8c2, State: Initialized, Role: FOLLOWER
I20260926 19:33:22.902899 10005 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42371 } }
I20260926 19:33:22.903134 10005 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:22.903280 10005 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:22.903469 10005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:22.904263 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:22.882899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.6.215.254:42371)
0926 19:33:22.883756 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:22.883769 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:22.883811 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:33:22.885600 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:22.885661 (+ 61us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:22.886072 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:22.886464 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:22.887956 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:22.889362 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:22.891097 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:22.891751 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:22.891960 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:22.896402 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:22.896411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:22.896430 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:22.896465 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:22.899717 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:22.900536 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:22.900543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:22.900550 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:22.900734 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:22.901167 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:22.901170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:22.901172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:22.901730 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:22.901981 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:22.902532 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 19:33:22.903012 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":382,"threads_started":1}
I20260926 19:33:22.905942 10005 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42371 } }
I20260926 19:33:22.906453 10005 leader_election.cc:302] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6297cf45d4249bf8d13582a20a0f8c2; no voters:
I20260926 19:33:22.907120 10005 leader_election.cc:288] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:22.907238 10010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:22.908382 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:33:22.909390 10005 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:22.914165 10010 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [term 1 LEADER]: Becoming Leader. State: Replica: b6297cf45d4249bf8d13582a20a0f8c2, State: Running, Role: LEADER
I20260926 19:33:22.914985 10010 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42371 } }
I20260926 19:33:22.918684 10012 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6297cf45d4249bf8d13582a20a0f8c2. Latest consensus state: current_term: 1 leader_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42371 } } }
I20260926 19:33:22.918776 10011 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6297cf45d4249bf8d13582a20a0f8c2" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42371 } } }
I20260926 19:33:22.919130 10011 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:22.919130 10012 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:22.919960 10018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:22.935626 10018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:22.967849 10018 catalog_manager.cc:1347] Generated new cluster ID: 33bb80be7ed64c5da262be6d190142fb
I20260926 19:33:22.967974 10018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:22.972658 10033 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:33:22.985679 10018 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:22.987505 10018 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:22.991004 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:22.976929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.6.215.254:42371)
0926 19:33:22.977174 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:22.977183 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:22.981445 (+ 4262us) server_negotiation.cc:407] Connection header received
0926 19:33:22.981659 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:22.981664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:22.981744 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:22.981859 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:22.982915 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:22.983666 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:22.987742 (+ 4076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:22.988176 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:22.988332 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:22.989512 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:22.989517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:22.989520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:22.990153 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:22.990211 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:22.990239 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:22.990363 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:22.990516 (+ 153us) server_negotiation.cc:299] Negotiation successful
0926 19:33:22.990643 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 19:33:23.005952 10018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6297cf45d4249bf8d13582a20a0f8c2: Generated new TSK 0
I20260926 19:33:23.006857 10018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:33:23.017684 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 9938
I20260926 19:33:23.018865 10000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:23.108412 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 9938
2026-09-26T19:33:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1943 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:23 dist-test-slave-phbh krb5kdc[10039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:23 dist-test-slave-phbh krb5kdc[10039](info): set up 2 sockets
Sep 26 19:33:23 dist-test-slave-phbh krb5kdc[10039](info): commencing operation
krb5kdc: starting...
W20260926 19:33:24.189878 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:24 dist-test-slave-phbh krb5kdc[10039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:24.384851 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:32825
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:32825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:24.704535 10052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:24.704928 10052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:24.705024 10052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:24.712184 10052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:24.712288 10052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:24.712332 10052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:24.712388 10052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:24.712431 10052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:24.722486 10052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:32825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:32825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:24.723740 10052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:24.724987 10052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:24.736542 10058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:24.736688 10060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:24.736588 10057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:24.737679 10052 server_base.cc:1034] running on GCE node
I20260926 19:33:24.738442 10052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:24.739861 10052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:24.741077 10052 hybrid_clock.cc:648] HybridClock initialized: now 1790451204741036 us; error 27 us; skew 500 ppm
I20260926 19:33:24.741534 10052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:24 dist-test-slave-phbh krb5kdc[10039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451204, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:24.744664 10052 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:24.746313 10052 webserver.cc:460] Webserver started at http://127.6.215.254:45699/ using document root <none> and password file <none>
I20260926 19:33:24.746951 10052 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:24.747049 10052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:24.747299 10052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:24.750293 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "8d45c21176eb4d7fa32c8ad1da571155"
format_stamp: "Formatted at 2026-09-26 19:33:24 on dist-test-slave-phbh"
server_key: "fc7d7ecc369d8a87ca415c78bc99dfc0"
server_key_iv: "f0a65ac9ed46616c4e86c58aef003765"
server_key_version: "encryptionkey@0"
I20260926 19:33:24.750994 10052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "8d45c21176eb4d7fa32c8ad1da571155"
format_stamp: "Formatted at 2026-09-26 19:33:24 on dist-test-slave-phbh"
server_key: "fc7d7ecc369d8a87ca415c78bc99dfc0"
server_key_iv: "f0a65ac9ed46616c4e86c58aef003765"
server_key_version: "encryptionkey@0"
I20260926 19:33:24.756084 10052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:33:24.759599 10067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:24.761080 10052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:24.761256 10052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "8d45c21176eb4d7fa32c8ad1da571155"
format_stamp: "Formatted at 2026-09-26 19:33:24 on dist-test-slave-phbh"
server_key: "fc7d7ecc369d8a87ca415c78bc99dfc0"
server_key_iv: "f0a65ac9ed46616c4e86c58aef003765"
server_key_version: "encryptionkey@0"
I20260926 19:33:24.761458 10052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:24.776693 10052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:24.781159 10052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:24.781464 10052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:24.799960 10052 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:32825
I20260926 19:33:24.799985 10118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:32825 every 8 connection(s)
I20260926 19:33:24.801342 10052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:24.803316 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10052
I20260926 19:33:24.803534 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:24.804159 7007 external_mini_cluster.cc:1428] Setting key d65754e61cb7a0ade06b765296b3f5ea
I20260926 19:33:24.805158 10119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 19:33:24 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:24.818619 10119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155: Bootstrap starting.
I20260926 19:33:24.821717 10119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:24.822758 10119 log.cc:824] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:24.825608 10119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155: No bootstrap required, opened a new log
I20260926 19:33:24.826732 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:24.806172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38384 (local address 127.6.215.254:32825)
0926 19:33:24.806925 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:24.806937 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:24.806975 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:33:24.808491 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:24.808513 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:24.808977 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:24.809379 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:24.810762 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:24.812069 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:24.813880 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:24.814563 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:24.814695 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:24.819519 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:24.819527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:24.819543 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:24.819582 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:24.822748 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:24.823456 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:24.823463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:24.823469 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:24.823632 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:24.823999 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:24.824005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:24.824011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:24.824522 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:24.824869 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:24.825305 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 19:33:24.825786 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":333,"threads_started":1}
I20260926 19:33:24.833724 10119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d45c21176eb4d7fa32c8ad1da571155" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 32825 } }
I20260926 19:33:24.834049 10119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:24.834141 10119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d45c21176eb4d7fa32c8ad1da571155, State: Initialized, Role: FOLLOWER
I20260926 19:33:24.834750 10119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d45c21176eb4d7fa32c8ad1da571155" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 32825 } }
I20260926 19:33:24.834970 10119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:24.835111 10119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:24.835286 10119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:24.837635 10119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d45c21176eb4d7fa32c8ad1da571155" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 32825 } }
I20260926 19:33:24.838121 10119 leader_election.cc:302] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d45c21176eb4d7fa32c8ad1da571155; no voters:
I20260926 19:33:24.838677 10119 leader_election.cc:288] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:24.838869 10124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:24.839632 10124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [term 1 LEADER]: Becoming Leader. State: Replica: 8d45c21176eb4d7fa32c8ad1da571155, State: Running, Role: LEADER
I20260926 19:33:24.840155 10124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d45c21176eb4d7fa32c8ad1da571155" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 32825 } }
I20260926 19:33:24.840754 10119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:24.842789 10125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d45c21176eb4d7fa32c8ad1da571155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d45c21176eb4d7fa32c8ad1da571155" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 32825 } } }
I20260926 19:33:24.843129 10125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:24.843624 10126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d45c21176eb4d7fa32c8ad1da571155. Latest consensus state: current_term: 1 leader_uuid: "8d45c21176eb4d7fa32c8ad1da571155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d45c21176eb4d7fa32c8ad1da571155" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 32825 } } }
I20260926 19:33:24.843900 10126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:24.845577 10131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:24.851398 10131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:24.860067 10131 catalog_manager.cc:1347] Generated new cluster ID: a29d91e566e844ecb79b1942030e6c5a
I20260926 19:33:24.860200 10131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:24.871805 10131 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:24.873679 10131 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:24.888221 10131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d45c21176eb4d7fa32c8ad1da571155: Generated new TSK 0
I20260926 19:33:24.889349 10131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:24.944733 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:32825
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:25.269757 10145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:25.270100 10145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:25.270273 10145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:25.277680 10145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:25.277846 10145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:25.278016 10145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:25.287963 10145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:25.289384 10145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:25.290721 10145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:25.303649 10150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:25.303752 10153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:25.303776 10151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:25.304565 10145 server_base.cc:1034] running on GCE node
I20260926 19:33:25.305224 10145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:25.306156 10145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:25.307381 10145 hybrid_clock.cc:648] HybridClock initialized: now 1790451205307322 us; error 56 us; skew 500 ppm
I20260926 19:33:25.307694 10145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:25 dist-test-slave-phbh krb5kdc[10039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451205, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:25.310747 10145 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:25.312182 10145 webserver.cc:460] Webserver started at http://127.6.215.193:38183/ using document root <none> and password file <none>
I20260926 19:33:25.312925 10145 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:25.313071 10145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:25.313378 10145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:25.316205 10145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "6bcbc56f074a431d8f2ff8ba6b3ded60"
format_stamp: "Formatted at 2026-09-26 19:33:25 on dist-test-slave-phbh"
server_key: "6a1fbfc9ed7a2fd812cacdc75c0d5777"
server_key_iv: "bacda4c31a7f73b4a9378c53acbbe65c"
server_key_version: "encryptionkey@0"
I20260926 19:33:25.317051 10145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "6bcbc56f074a431d8f2ff8ba6b3ded60"
format_stamp: "Formatted at 2026-09-26 19:33:25 on dist-test-slave-phbh"
server_key: "6a1fbfc9ed7a2fd812cacdc75c0d5777"
server_key_iv: "bacda4c31a7f73b4a9378c53acbbe65c"
server_key_version: "encryptionkey@0"
I20260926 19:33:25.322149 10145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 19:33:25.325816 10160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:25.327322 10145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:33:25.327491 10145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "6bcbc56f074a431d8f2ff8ba6b3ded60"
format_stamp: "Formatted at 2026-09-26 19:33:25 on dist-test-slave-phbh"
server_key: "6a1fbfc9ed7a2fd812cacdc75c0d5777"
server_key_iv: "bacda4c31a7f73b4a9378c53acbbe65c"
server_key_version: "encryptionkey@0"
I20260926 19:33:25.327699 10145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:25.351634 10145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:25.355665 10145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:25.355984 10145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:25.356962 10145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:25.358258 10145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:25.358358 10145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:25.358507 10145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:25.358587 10145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:25.392112 10145 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:41191
I20260926 19:33:25.392159 10272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:41191 every 8 connection(s)
I20260926 19:33:25.393558 10145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:25.395520 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10145
I20260926 19:33:25.395736 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:25.396551 7007 external_mini_cluster.cc:1428] Setting key 403595e3c75005f238e0e7ed76277d5d
Sep 26 19:33:25 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451205, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
I20260926 19:33:25.419984 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:25.396182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:43115 (local address 127.6.215.254:32825)
0926 19:33:25.396633 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:25.396638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:25.397997 (+ 1359us) server_negotiation.cc:407] Connection header received
0926 19:33:25.400684 (+ 2687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:25.400688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:25.400760 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:25.400899 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:25.402986 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.403668 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.405525 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.406293 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.406449 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:25.412311 (+ 5862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:25.412321 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:25.412326 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:25.412373 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:25.416462 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:25.417564 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:25.417569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:25.417571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:25.417709 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:25.418200 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:25.418207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:25.418213 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:25.418504 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:25.418615 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:25.419448 (+ 833us) server_negotiation.cc:299] Negotiation successful
0926 19:33:25.419630 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":241,"threads_started":1}
I20260926 19:33:25.421582 10276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:25.396953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:32825 (local address 127.6.215.193:43115)
0926 19:33:25.397837 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:25.397884 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:25.400362 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:25.401182 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:25.401195 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:25.401971 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:25.402806 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:25.402836 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.403849 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.403857 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:25.405377 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:25.405388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.406496 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.406501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:25.406703 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:25.407596 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:25.407633 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:25.411996 (+ 4363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:25.416688 (+ 4692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:25.416699 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:25.416733 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:25.417414 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:25.417821 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:25.417824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:25.417827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:25.418018 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:25.418621 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:25.418628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:25.418969 (+ 341us) client_negotiation.cc:769] Sending connection context
0926 19:33:25.419307 (+ 338us) client_negotiation.cc:240] Negotiation successful
0926 19:33:25.419868 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":383,"threads_started":1}
I20260926 19:33:25.423206 10273 heartbeater.cc:347] Connected to a master server at 127.6.215.254:32825
I20260926 19:33:25.423544 10273 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:25.424471 10273 heartbeater.cc:511] Master 127.6.215.254:32825 requested a full tablet report, sending...
I20260926 19:33:25.426756 10084 ts_manager.cc:194] Registered new tserver with Master: 6bcbc56f074a431d8f2ff8ba6b3ded60 (127.6.215.193:41191)
I20260926 19:33:25.428181 10084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:43115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:25.444084 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:32825
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:25.761210 10279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:25.761524 10279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:25.761699 10279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:25.768913 10279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:25.769057 10279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:25.769243 10279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:25.778966 10279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:25.780269 10279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:25.781669 10279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:25.794828 10284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:25.794854 10287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:25.794901 10285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:25.795639 10279 server_base.cc:1034] running on GCE node
I20260926 19:33:25.796293 10279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:25.797334 10279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:25.798619 10279 hybrid_clock.cc:648] HybridClock initialized: now 1790451205798551 us; error 51 us; skew 500 ppm
I20260926 19:33:25.798952 10279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:25 dist-test-slave-phbh krb5kdc[10039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451205, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:25.802366 10279 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:25.803869 10279 webserver.cc:460] Webserver started at http://127.6.215.194:36119/ using document root <none> and password file <none>
I20260926 19:33:25.804534 10279 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:25.804682 10279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:25.805011 10279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:25.808132 10279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "a0ab3cd5c31c42fd8cfed2709cff17b9"
format_stamp: "Formatted at 2026-09-26 19:33:25 on dist-test-slave-phbh"
server_key: "e5e446e3e9a86a63a326f7b8c15bf3c6"
server_key_iv: "5e92f8fcd5a04a8a80c22e4319f03ac2"
server_key_version: "encryptionkey@0"
I20260926 19:33:25.809077 10279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "a0ab3cd5c31c42fd8cfed2709cff17b9"
format_stamp: "Formatted at 2026-09-26 19:33:25 on dist-test-slave-phbh"
server_key: "e5e446e3e9a86a63a326f7b8c15bf3c6"
server_key_iv: "5e92f8fcd5a04a8a80c22e4319f03ac2"
server_key_version: "encryptionkey@0"
I20260926 19:33:25.814220 10279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 19:33:25.817696 10294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:25.818981 10279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:33:25.819214 10279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "a0ab3cd5c31c42fd8cfed2709cff17b9"
format_stamp: "Formatted at 2026-09-26 19:33:25 on dist-test-slave-phbh"
server_key: "e5e446e3e9a86a63a326f7b8c15bf3c6"
server_key_iv: "5e92f8fcd5a04a8a80c22e4319f03ac2"
server_key_version: "encryptionkey@0"
I20260926 19:33:25.819417 10279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:25.835564 10279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:25.840046 10279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:25.840342 10279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:25.841408 10279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:25.842727 10279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:25.842798 10279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:25.842962 10279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:25.843012 10279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:25.875885 10279 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:38411
I20260926 19:33:25.875921 10406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:38411 every 8 connection(s)
I20260926 19:33:25.877249 10279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:25.882321 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10279
I20260926 19:33:25.882510 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:25.883038 7007 external_mini_cluster.cc:1428] Setting key cfce6cc9c3824049890cdd92eb71d9ec
Sep 26 19:33:25 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451205, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:25.900347 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:25.879806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:53201 (local address 127.6.215.254:32825)
0926 19:33:25.879970 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:25.879975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:25.881220 (+ 1245us) server_negotiation.cc:407] Connection header received
0926 19:33:25.883613 (+ 2393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:25.883622 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:25.883733 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:25.883857 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:25.885732 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.886429 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.889064 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.889448 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.889542 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:25.893440 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:25.893449 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:25.893457 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:25.893500 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:25.897005 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:25.897704 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:25.897713 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:25.897719 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:25.897869 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:25.898280 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:25.898289 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:25.898295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:25.898679 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:25.898841 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:25.899624 (+ 783us) server_negotiation.cc:299] Negotiation successful
0926 19:33:25.899898 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 19:33:25.901021 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:25.880295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:32825 (local address 127.6.215.194:53201)
0926 19:33:25.881057 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:25.881100 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:25.883271 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:25.884151 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:25.884160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:25.884739 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:25.885551 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:25.885578 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.886597 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.886605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:25.888930 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:25.888948 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:25.889681 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:25.889691 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:25.890027 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:25.890816 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:25.890838 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:25.893200 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:25.897140 (+ 3940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:25.897147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:25.897161 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:25.897564 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:25.897975 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:25.897978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:25.897980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:25.898157 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:25.898817 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:25.898823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:25.899237 (+ 414us) client_negotiation.cc:769] Sending connection context
0926 19:33:25.899459 (+ 222us) client_negotiation.cc:240] Negotiation successful
0926 19:33:25.899841 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":284,"threads_started":1}
I20260926 19:33:25.902415 10407 heartbeater.cc:347] Connected to a master server at 127.6.215.254:32825
I20260926 19:33:25.902770 10407 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:33:25.903750 10407 heartbeater.cc:511] Master 127.6.215.254:32825 requested a full tablet report, sending...
I20260926 19:33:25.905206 10084 ts_manager.cc:194] Registered new tserver with Master: a0ab3cd5c31c42fd8cfed2709cff17b9 (127.6.215.194:38411)
I20260926 19:33:25.906016 10084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:53201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:25.928359 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:32825
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:26.251537 10412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:26.251811 10412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:26.251984 10412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:26.259135 10412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:26.259249 10412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:26.259466 10412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:26.269212 10412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:32825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:26.270532 10412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:26.271853 10412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:26.284906 10417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:26.285029 10418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:26.285055 10420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:26.285982 10412 server_base.cc:1034] running on GCE node
I20260926 19:33:26.286625 10412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:26.287546 10412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:26.288870 10412 hybrid_clock.cc:648] HybridClock initialized: now 1790451206288812 us; error 57 us; skew 500 ppm
I20260926 19:33:26.289206 10412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451206, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:26.292225 10412 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:26.293810 10412 webserver.cc:460] Webserver started at http://127.6.215.195:35453/ using document root <none> and password file <none>
I20260926 19:33:26.294499 10412 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:26.294644 10412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:26.294950 10412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:26.297890 10412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "1ce53c1c05464098987f2327ee19bbcd"
format_stamp: "Formatted at 2026-09-26 19:33:26 on dist-test-slave-phbh"
server_key: "b5cc42c56d349af8affbab061c9489b2"
server_key_iv: "a273e4297aff1b734839206eb40ee50b"
server_key_version: "encryptionkey@0"
I20260926 19:33:26.298664 10412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "1ce53c1c05464098987f2327ee19bbcd"
format_stamp: "Formatted at 2026-09-26 19:33:26 on dist-test-slave-phbh"
server_key: "b5cc42c56d349af8affbab061c9489b2"
server_key_iv: "a273e4297aff1b734839206eb40ee50b"
server_key_version: "encryptionkey@0"
I20260926 19:33:26.303941 10412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 19:33:26.307345 10427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:26.308701 10412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:33:26.308910 10412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "1ce53c1c05464098987f2327ee19bbcd"
format_stamp: "Formatted at 2026-09-26 19:33:26 on dist-test-slave-phbh"
server_key: "b5cc42c56d349af8affbab061c9489b2"
server_key_iv: "a273e4297aff1b734839206eb40ee50b"
server_key_version: "encryptionkey@0"
I20260926 19:33:26.309114 10412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:26.327049 10412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:26.331394 10412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:26.331743 10412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:26.332729 10412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:26.334266 10412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:26.334364 10412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:26.334517 10412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:26.334586 10412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:26.367746 10412 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:44993
I20260926 19:33:26.367794 10539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:44993 every 8 connection(s)
I20260926 19:33:26.369103 10412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:26.377346 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10412
I20260926 19:33:26.377532 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:26.377991 7007 external_mini_cluster.cc:1428] Setting key 9fe668ef471eb0d285d1812c36bea398
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451206, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:26.389528 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:26.371605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:41735 (local address 127.6.215.254:32825)
0926 19:33:26.371801 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:26.371806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:26.373083 (+ 1277us) server_negotiation.cc:407] Connection header received
0926 19:33:26.374952 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:26.374958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:26.375060 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:26.375173 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:26.376732 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.377594 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.379606 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.380176 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.380310 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:26.383870 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:26.383877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:26.383882 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:26.383922 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:26.386480 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.387219 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.387227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.387232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.387359 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.387795 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.387800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.387804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.388063 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:26.388226 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:26.388952 (+ 726us) server_negotiation.cc:299] Negotiation successful
0926 19:33:26.389170 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 19:33:26.390640 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:26.372086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:32825 (local address 127.6.215.195:41735)
0926 19:33:26.372911 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:26.372950 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:26.374669 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:26.375415 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:26.375423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:26.375925 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:26.376476 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:26.376492 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.377772 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.377785 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:26.379467 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:26.379479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.380397 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.380406 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:26.380737 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:26.381425 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:26.381447 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:26.383645 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:26.386635 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:26.386645 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:26.386663 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:26.387088 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:26.387471 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:26.387477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:26.387481 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:26.387676 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:26.388211 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:26.388219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:26.388561 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 19:33:26.388873 (+ 312us) client_negotiation.cc:240] Negotiation successful
0926 19:33:26.389395 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":297,"threads_started":1}
I20260926 19:33:26.392138 10540 heartbeater.cc:347] Connected to a master server at 127.6.215.254:32825
I20260926 19:33:26.392407 10540 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:26.393371 10540 heartbeater.cc:511] Master 127.6.215.254:32825 requested a full tablet report, sending...
I20260926 19:33:26.394930 10084 ts_manager.cc:194] Registered new tserver with Master: 1ce53c1c05464098987f2327ee19bbcd (127.6.215.195:44993)
I20260926 19:33:26.396034 10084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:41735
I20260926 19:33:26.405803 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:33:26.431200 10273 heartbeater.cc:503] Master 127.6.215.254:32825 was elected leader, sending a full tablet report...
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451206, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451206, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:26.453042 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:26.439770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.6.215.254:32825)
0926 19:33:26.439942 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:26.439946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:26.440411 (+ 465us) server_negotiation.cc:407] Connection header received
0926 19:33:26.440548 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:26.440552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:26.440613 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:26.440701 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:26.442136 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.442868 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.444112 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.444505 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.444595 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:26.448234 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:26.448239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:26.448242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:26.448274 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:26.450782 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.451353 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.451356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.451359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.451495 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.451826 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.451830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.451832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.452117 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:26.452293 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:26.452511 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 19:33:26.452657 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260926 19:33:26.453555 10084 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51158
I20260926 19:33:26.455657 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10145
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451206, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:26.470965 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:26.456736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.6.215.193:41191)
0926 19:33:26.457415 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:26.457421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:26.457445 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:33:26.457502 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:26.457508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:26.457710 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:26.457844 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:26.458959 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.459947 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.461155 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.461697 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.461798 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:26.465639 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:26.465650 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:26.465664 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:26.465698 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:26.468296 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.469080 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.469087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.469089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.469226 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.469565 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.469568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.469570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.469883 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:26.470094 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:26.470388 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 19:33:26.470575 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":332,"threads_started":1}
W20260926 19:33:26.472186 10268 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58192
W20260926 19:33:26.473372 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10145: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:26.473590 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10145
I20260926 19:33:26.489370 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10279
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451206, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:33:26.504594 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:26.490257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.6.215.194:38411)
0926 19:33:26.490829 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:26.490836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:26.490863 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:33:26.490922 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:26.490929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:26.491145 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:26.491287 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:26.492421 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.493483 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.494644 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.495167 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.495271 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:26.499236 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:26.499248 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:26.499262 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:26.499293 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:26.502236 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.502801 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.502808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.502811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.502926 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.503249 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.503252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.503254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.503567 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:26.503784 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:26.504022 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 19:33:26.504215 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":327,"threads_started":1}
W20260926 19:33:26.505926 10402 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51908
W20260926 19:33:26.506969 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10279: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:26.507133 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10279
I20260926 19:33:26.522410 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10412
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451206, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:26.538856 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:26.523426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37192 (local address 127.6.215.195:44993)
0926 19:33:26.524031 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:26.524038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:26.524066 (+ 28us) server_negotiation.cc:407] Connection header received
0926 19:33:26.524124 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:26.524130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:26.524342 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:26.524520 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:26.525856 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.527102 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.528375 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:26.529011 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:26.529238 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:26.533352 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:26.533362 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:26.533377 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:26.533412 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:26.535996 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.536651 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.536662 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.536679 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.536921 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:26.537258 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:26.537264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:26.537267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:26.537598 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:26.537903 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:26.538241 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 19:33:26.538513 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":320,"threads_started":1}
W20260926 19:33:26.540110 10535 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37192
W20260926 19:33:26.541275 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10412: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:33:26.541452 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10412
I20260926 19:33:26.556715 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10052
I20260926 19:33:26.557956 10114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:26.659439 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10052
2026-09-26T19:33:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3555 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10557](info): set up 2 sockets
Sep 26 19:33:26 dist-test-slave-phbh krb5kdc[10557](info): commencing operation
krb5kdc: starting...
W20260926 19:33:27.753925 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.008s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:27 dist-test-slave-phbh krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:27.949517 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:44837
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:44837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:28.269433 10570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:28.269721 10570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:28.269886 10570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:28.277194 10570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:28.277349 10570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:28.277431 10570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:28.277527 10570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:28.277592 10570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:28.287918 10570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:44837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:44837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:28.289525 10570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:28.290865 10570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:28.302934 10576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:28.303031 10575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:28.303010 10578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:28.303983 10570 server_base.cc:1034] running on GCE node
I20260926 19:33:28.304879 10570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:28.306396 10570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:28.307626 10570 hybrid_clock.cc:648] HybridClock initialized: now 1790451208307576 us; error 39 us; skew 500 ppm
I20260926 19:33:28.307984 10570 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:28 dist-test-slave-phbh krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451208, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:28.314759 10570 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:28.316424 10570 webserver.cc:460] Webserver started at http://127.6.215.254:42501/ using document root <none> and password file <none>
I20260926 19:33:28.317237 10570 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:28.317389 10570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:28.317718 10570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:28.320729 10570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "54cd8c6cb447484a836a832aaecaef46"
format_stamp: "Formatted at 2026-09-26 19:33:28 on dist-test-slave-phbh"
server_key: "2c62ee479ff3a15de0787691d1d6320b"
server_key_iv: "2eb0e0b6a9c0d2a9317a351a587fe077"
server_key_version: "encryptionkey@0"
I20260926 19:33:28.321575 10570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "54cd8c6cb447484a836a832aaecaef46"
format_stamp: "Formatted at 2026-09-26 19:33:28 on dist-test-slave-phbh"
server_key: "2c62ee479ff3a15de0787691d1d6320b"
server_key_iv: "2eb0e0b6a9c0d2a9317a351a587fe077"
server_key_version: "encryptionkey@0"
I20260926 19:33:28.326745 10570 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:28.330348 10585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:28.331764 10570 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:33:28.331956 10570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "54cd8c6cb447484a836a832aaecaef46"
format_stamp: "Formatted at 2026-09-26 19:33:28 on dist-test-slave-phbh"
server_key: "2c62ee479ff3a15de0787691d1d6320b"
server_key_iv: "2eb0e0b6a9c0d2a9317a351a587fe077"
server_key_version: "encryptionkey@0"
I20260926 19:33:28.332201 10570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:28.370785 10570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:28.374954 10570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:28.375258 10570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:28.394495 10570 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:44837
I20260926 19:33:28.394532 10636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:44837 every 8 connection(s)
I20260926 19:33:28.395919 10570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:28.398300 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10570
I20260926 19:33:28.398520 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:28.399166 7007 external_mini_cluster.cc:1428] Setting key 0648c46db5d98b77ca525cbbfbfc1821
I20260926 19:33:28.399816 10637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:28.414219 10637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46: Bootstrap starting.
Sep 26 19:33:28 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:28.417711 10637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:28.418888 10637 log.cc:824] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:28.422043 10637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46: No bootstrap required, opened a new log
I20260926 19:33:28.423014 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:28.401176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.6.215.254:44837)
0926 19:33:28.401953 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:28.401965 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:28.402003 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:33:28.404251 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:28.404271 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:28.404662 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:28.405072 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:28.406976 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:28.408280 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:28.410069 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:28.410780 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:28.410910 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:28.415580 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:28.415588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:28.415603 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:28.415651 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:28.418751 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:28.419527 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:28.419534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:28.419540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:28.419716 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:28.420116 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:28.420120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:28.420122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:28.420743 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:28.421083 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:28.421531 (+ 448us) server_negotiation.cc:299] Negotiation successful
0926 19:33:28.421958 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":348,"threads_started":1}
I20260926 19:33:28.429793 10637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cd8c6cb447484a836a832aaecaef46" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44837 } }
I20260926 19:33:28.430199 10637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:28.430313 10637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54cd8c6cb447484a836a832aaecaef46, State: Initialized, Role: FOLLOWER
I20260926 19:33:28.430899 10637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cd8c6cb447484a836a832aaecaef46" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44837 } }
I20260926 19:33:28.431120 10637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:28.431252 10637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:28.431417 10637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:28.433768 10637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cd8c6cb447484a836a832aaecaef46" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44837 } }
I20260926 19:33:28.434279 10637 leader_election.cc:302] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54cd8c6cb447484a836a832aaecaef46; no voters:
I20260926 19:33:28.434881 10637 leader_election.cc:288] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:28.435024 10642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:28.435757 10642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [term 1 LEADER]: Becoming Leader. State: Replica: 54cd8c6cb447484a836a832aaecaef46, State: Running, Role: LEADER
I20260926 19:33:28.436301 10642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cd8c6cb447484a836a832aaecaef46" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44837 } }
I20260926 19:33:28.437160 10637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:28.439489 10643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54cd8c6cb447484a836a832aaecaef46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cd8c6cb447484a836a832aaecaef46" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44837 } } }
I20260926 19:33:28.439661 10644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54cd8c6cb447484a836a832aaecaef46. Latest consensus state: current_term: 1 leader_uuid: "54cd8c6cb447484a836a832aaecaef46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54cd8c6cb447484a836a832aaecaef46" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44837 } } }
I20260926 19:33:28.439973 10643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:28.440061 10644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:28.442060 10650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:28.447505 10650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:28.456192 10650 catalog_manager.cc:1347] Generated new cluster ID: 4a0e0c2627b84da0889eb5fd3e234296
I20260926 19:33:28.456342 10650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:28.479964 10650 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:28.481684 10650 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:28.493449 10650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54cd8c6cb447484a836a832aaecaef46: Generated new TSK 0
I20260926 19:33:28.494643 10650 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:28.561251 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:44837
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:28.893587 10663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:28.893874 10663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:28.894033 10663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:28.901986 10663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:28.902089 10663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:28.902297 10663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:28.912216 10663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:44837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:28.913547 10663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:28.914841 10663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:28.929297 10669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:28.929405 10668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:28.929471 10671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:28.930048 10663 server_base.cc:1034] running on GCE node
I20260926 19:33:28.930642 10663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:28.931619 10663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:28.932870 10663 hybrid_clock.cc:648] HybridClock initialized: now 1790451208932829 us; error 50 us; skew 500 ppm
I20260926 19:33:28.933182 10663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:28 dist-test-slave-phbh krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451208, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:28.936218 10663 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:28.944953 10663 webserver.cc:460] Webserver started at http://127.6.215.193:36537/ using document root <none> and password file <none>
I20260926 19:33:28.945770 10663 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:28.945931 10663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:28.946256 10663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:28.949225 10663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "933f71c3648445b18bbbe2082f244a7e"
format_stamp: "Formatted at 2026-09-26 19:33:28 on dist-test-slave-phbh"
server_key: "63c4e3864305936812908e86e34b4a20"
server_key_iv: "f7d99fddd9e6a9199c93d716154e21b2"
server_key_version: "encryptionkey@0"
I20260926 19:33:28.949990 10663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "933f71c3648445b18bbbe2082f244a7e"
format_stamp: "Formatted at 2026-09-26 19:33:28 on dist-test-slave-phbh"
server_key: "63c4e3864305936812908e86e34b4a20"
server_key_iv: "f7d99fddd9e6a9199c93d716154e21b2"
server_key_version: "encryptionkey@0"
I20260926 19:33:28.955051 10663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:28.958528 10678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:28.959797 10663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:33:28.959964 10663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "933f71c3648445b18bbbe2082f244a7e"
format_stamp: "Formatted at 2026-09-26 19:33:28 on dist-test-slave-phbh"
server_key: "63c4e3864305936812908e86e34b4a20"
server_key_iv: "f7d99fddd9e6a9199c93d716154e21b2"
server_key_version: "encryptionkey@0"
I20260926 19:33:28.960165 10663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:28.976643 10663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:28.980643 10663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:28.980993 10663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:28.981936 10663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:28.983274 10663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:28.983381 10663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:28.983536 10663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:28.983609 10663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:29.016467 10663 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:39783
I20260926 19:33:29.016515 10790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:39783 every 8 connection(s)
I20260926 19:33:29.017822 10663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:29.026517 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10663
I20260926 19:33:29.026737 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:29.027498 7007 external_mini_cluster.cc:1428] Setting key 49eec9ac692fb94238baa4acc961600a
Sep 26 19:33:29 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451208, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:29.039108 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:29.020245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:34705 (local address 127.6.215.254:44837)
0926 19:33:29.020656 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:29.020666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:29.021589 (+ 923us) server_negotiation.cc:407] Connection header received
0926 19:33:29.023347 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:29.023353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:29.023443 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:29.023563 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:29.024993 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.025694 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.027996 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.028542 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.028645 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:29.032120 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:29.032129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:29.032137 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:29.032189 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:29.035501 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:29.036517 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:29.036521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:29.036523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:29.036684 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:29.037212 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:29.037215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:29.037217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:29.037569 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:29.037680 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:29.038637 (+ 957us) server_negotiation.cc:299] Negotiation successful
0926 19:33:29.038846 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":227,"threads_started":1}
I20260926 19:33:29.040545 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:29.020693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:44837 (local address 127.6.215.193:34705)
0926 19:33:29.021425 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:29.021462 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:29.023077 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:29.023793 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:29.023801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:29.024288 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:29.024830 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:29.024846 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.025840 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.025846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:29.027831 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:29.027846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.028764 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.028791 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:29.029060 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:29.029832 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:29.029870 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:29.031911 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:29.035680 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:29.035690 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:29.035707 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:29.036400 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:29.036835 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:29.036842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:29.036849 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:29.037096 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:29.037744 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:29.037755 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:29.038222 (+ 467us) client_negotiation.cc:769] Sending connection context
0926 19:33:29.038525 (+ 303us) client_negotiation.cc:240] Negotiation successful
0926 19:33:29.039112 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":264,"threads_started":1}
I20260926 19:33:29.042150 10791 heartbeater.cc:347] Connected to a master server at 127.6.215.254:44837
I20260926 19:33:29.042484 10791 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:29.043205 10791 heartbeater.cc:511] Master 127.6.215.254:44837 requested a full tablet report, sending...
I20260926 19:33:29.045394 10602 ts_manager.cc:194] Registered new tserver with Master: 933f71c3648445b18bbbe2082f244a7e (127.6.215.193:39783)
I20260926 19:33:29.046787 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:34705
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:29.071928 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:44837
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:29.388002 10797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:29.388298 10797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:29.388463 10797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:29.395838 10797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:29.396000 10797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:29.396195 10797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:29.406010 10797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:44837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:29.407651 10797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:29.409672 10797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:29.424580 10802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:29.424559 10803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:29.424540 10805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:29.425503 10797 server_base.cc:1034] running on GCE node
I20260926 19:33:29.426122 10797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:29.427111 10797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:29.428370 10797 hybrid_clock.cc:648] HybridClock initialized: now 1790451209428296 us; error 54 us; skew 500 ppm
I20260926 19:33:29.428686 10797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:29 dist-test-slave-phbh krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451209, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:29.432013 10797 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:29.433537 10797 webserver.cc:460] Webserver started at http://127.6.215.194:45253/ using document root <none> and password file <none>
I20260926 19:33:29.434213 10797 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:29.434357 10797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:29.434659 10797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:29.438128 10797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "01f476ddc6714b8aabbd7a7b92187fb4"
format_stamp: "Formatted at 2026-09-26 19:33:29 on dist-test-slave-phbh"
server_key: "3b2501b48938b42477e732e71db42841"
server_key_iv: "55dd9cfca07dbe2e6ecd57e0f9ba0318"
server_key_version: "encryptionkey@0"
I20260926 19:33:29.438956 10797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "01f476ddc6714b8aabbd7a7b92187fb4"
format_stamp: "Formatted at 2026-09-26 19:33:29 on dist-test-slave-phbh"
server_key: "3b2501b48938b42477e732e71db42841"
server_key_iv: "55dd9cfca07dbe2e6ecd57e0f9ba0318"
server_key_version: "encryptionkey@0"
I20260926 19:33:29.444074 10797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:33:29.447506 10812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:29.448904 10797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:29.449162 10797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "01f476ddc6714b8aabbd7a7b92187fb4"
format_stamp: "Formatted at 2026-09-26 19:33:29 on dist-test-slave-phbh"
server_key: "3b2501b48938b42477e732e71db42841"
server_key_iv: "55dd9cfca07dbe2e6ecd57e0f9ba0318"
server_key_version: "encryptionkey@0"
I20260926 19:33:29.449340 10797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:29.480199 10797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:29.484299 10797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:29.484613 10797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:29.485620 10797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:29.486977 10797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:29.487066 10797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:29.487237 10797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:29.487296 10797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:29.520648 10797 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:45109
I20260926 19:33:29.520668 10924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:45109 every 8 connection(s)
I20260926 19:33:29.521991 10797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:29.530393 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10797
I20260926 19:33:29.530625 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:29.531260 7007 external_mini_cluster.cc:1428] Setting key 110f2b9ea3129e0e5dcd18cd379e026b
Sep 26 19:33:29 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451209, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:29.542943 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:29.524584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:51465 (local address 127.6.215.254:44837)
0926 19:33:29.524763 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:29.524814 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:29.525942 (+ 1128us) server_negotiation.cc:407] Connection header received
0926 19:33:29.527714 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:29.527719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:29.527809 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:29.527922 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:29.529468 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.530200 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.532083 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.532586 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.532731 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:29.536339 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:29.536348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:29.536355 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:29.536398 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:29.539229 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:29.540213 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:29.540217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:29.540219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:29.540346 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:29.540900 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:29.540908 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:29.540915 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:29.541311 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:29.541468 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:29.542498 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0926 19:33:29.542670 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:33:29.544421 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:29.525127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:44837 (local address 127.6.215.194:51465)
0926 19:33:29.525791 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:29.525828 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:29.527440 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:29.528162 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:29.528171 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:29.528662 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:29.529295 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:29.529313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.530365 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.530369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:29.531956 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:29.531973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:29.532838 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:29.532845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:29.533138 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:29.533894 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:29.533917 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:29.536081 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:29.539377 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:29.539389 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:29.539411 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:29.540036 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:29.540459 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:29.540467 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:29.540473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:29.540748 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:29.541557 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:29.541567 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:29.542040 (+ 473us) client_negotiation.cc:769] Sending connection context
0926 19:33:29.542365 (+ 325us) client_negotiation.cc:240] Negotiation successful
0926 19:33:29.542934 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":270,"threads_started":1}
I20260926 19:33:29.546090 10925 heartbeater.cc:347] Connected to a master server at 127.6.215.254:44837
I20260926 19:33:29.546428 10925 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:29.547322 10925 heartbeater.cc:511] Master 127.6.215.254:44837 requested a full tablet report, sending...
I20260926 19:33:29.549072 10602 ts_manager.cc:194] Registered new tserver with Master: 01f476ddc6714b8aabbd7a7b92187fb4 (127.6.215.194:45109)
I20260926 19:33:29.550298 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:51465
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:29.579705 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:44837
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:29.896196 10930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:29.896524 10930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:29.896629 10930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:29.903968 10930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:29.904076 10930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:29.904196 10930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:29.913977 10930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:44837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:29.915203 10930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:29.916441 10930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:29.929430 10938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:29.929539 10935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:29.929567 10936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:29.930541 10930 server_base.cc:1034] running on GCE node
I20260926 19:33:29.931166 10930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:29.932192 10930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:29.933414 10930 hybrid_clock.cc:648] HybridClock initialized: now 1790451209933363 us; error 65 us; skew 500 ppm
I20260926 19:33:29.933733 10930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:29 dist-test-slave-phbh krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451209, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:29.937024 10930 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:29.938478 10930 webserver.cc:460] Webserver started at http://127.6.215.195:34133/ using document root <none> and password file <none>
I20260926 19:33:29.939216 10930 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:29.939321 10930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:29.939565 10930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:29.943331 10930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "8c0d47440f494ea48139b93e1dca619a"
format_stamp: "Formatted at 2026-09-26 19:33:29 on dist-test-slave-phbh"
server_key: "108acd02a7184951a72889f559af91bb"
server_key_iv: "a40c5f4a35885335cf27d7345efa2a5c"
server_key_version: "encryptionkey@0"
I20260926 19:33:29.944242 10930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "8c0d47440f494ea48139b93e1dca619a"
format_stamp: "Formatted at 2026-09-26 19:33:29 on dist-test-slave-phbh"
server_key: "108acd02a7184951a72889f559af91bb"
server_key_iv: "a40c5f4a35885335cf27d7345efa2a5c"
server_key_version: "encryptionkey@0"
I20260926 19:33:29.949855 10930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 19:33:29.953501 10945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:29.954844 10930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:33:29.955025 10930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "8c0d47440f494ea48139b93e1dca619a"
format_stamp: "Formatted at 2026-09-26 19:33:29 on dist-test-slave-phbh"
server_key: "108acd02a7184951a72889f559af91bb"
server_key_iv: "a40c5f4a35885335cf27d7345efa2a5c"
server_key_version: "encryptionkey@0"
I20260926 19:33:29.955255 10930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:29.987341 10930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:29.991312 10930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:29.991611 10930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:29.992578 10930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:29.993873 10930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:29.993986 10930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:29.994153 10930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:29.994227 10930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:30.026857 10930 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:43669
I20260926 19:33:30.026903 11057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:43669 every 8 connection(s)
I20260926 19:33:30.028169 10930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:30.038228 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 10930
I20260926 19:33:30.038435 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:30.038851 7007 external_mini_cluster.cc:1428] Setting key 3aa0e7288d32637b8d02a3df7385bb91
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451209, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:30.047739 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:30.030736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:43797 (local address 127.6.215.254:44837)
0926 19:33:30.030905 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:30.030910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:30.032032 (+ 1122us) server_negotiation.cc:407] Connection header received
0926 19:33:30.033906 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:30.033911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:30.034030 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:30.034148 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:30.035638 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.036447 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.038142 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.038678 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.038803 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:30.042739 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:30.042744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:30.042747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:30.042777 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:30.045190 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.045874 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.045877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.045880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.046005 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.046363 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.046366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.046368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.046617 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:30.046721 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:30.047325 (+ 604us) server_negotiation.cc:299] Negotiation successful
0926 19:33:30.047482 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:33:30.048828 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:30.031187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:44837 (local address 127.6.215.195:43797)
0926 19:33:30.031849 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:30.031888 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:30.033632 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:30.034376 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:30.034385 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:30.034872 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:30.035409 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:30.035457 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.036609 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.036615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:30.038002 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:30.038014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.038911 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.038918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:30.039140 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:30.039918 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:30.039948 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:30.042540 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:30.045312 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:30.045319 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:30.045332 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:30.045764 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:30.046094 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:30.046096 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:30.046099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:30.046268 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:30.046719 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:30.046725 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:30.047030 (+ 305us) client_negotiation.cc:769] Sending connection context
0926 19:33:30.047247 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 19:33:30.047621 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":264,"threads_started":1}
I20260926 19:33:30.049890 10791 heartbeater.cc:503] Master 127.6.215.254:44837 was elected leader, sending a full tablet report...
I20260926 19:33:30.050383 11058 heartbeater.cc:347] Connected to a master server at 127.6.215.254:44837
I20260926 19:33:30.050730 11058 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:30.051579 11058 heartbeater.cc:511] Master 127.6.215.254:44837 requested a full tablet report, sending...
I20260926 19:33:30.053068 10602 ts_manager.cc:194] Registered new tserver with Master: 8c0d47440f494ea48139b93e1dca619a (127.6.215.195:43669)
I20260926 19:33:30.054128 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:43797
I20260926 19:33:30.054423 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:30.102872 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:30.087947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.6.215.193:39783)
0926 19:33:30.088579 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:30.088586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:30.088612 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:33:30.088684 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:30.088690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:30.088980 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:30.089112 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:30.090455 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.091467 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.092638 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.093279 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.093384 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:30.097327 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:30.097341 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:30.097357 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:30.097393 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:30.100147 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.101129 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.101136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.101138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.101269 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.101608 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.101611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.101613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.101939 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:30.102177 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:30.102407 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 19:33:30.102597 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":328,"threads_started":1}
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:30.120455 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:30.106976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51528 (local address 127.6.215.254:44837)
0926 19:33:30.107219 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:30.107226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:30.107455 (+ 229us) server_negotiation.cc:407] Connection header received
0926 19:33:30.107639 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:30.107644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:30.107720 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:30.107821 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:30.109049 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.109765 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.111111 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.111535 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.111671 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:30.115442 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:30.115449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:30.115453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:30.115493 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:30.117911 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.118537 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.118544 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.118547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.118708 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.119060 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.119066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.119070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.119389 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:30.119620 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:30.119906 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 19:33:30.120111 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 19:33:30.121078 10602 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51528
I20260926 19:33:30.122044 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10663
I20260926 19:33:30.134214 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:30.123021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.6.215.193:39783)
0926 19:33:30.123216 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:30.123222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:30.123451 (+ 229us) server_negotiation.cc:407] Connection header received
0926 19:33:30.123608 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:30.123611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:30.123680 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:30.123754 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:30.125102 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.125750 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.126853 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.127264 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.127354 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:30.129494 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:30.129499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:30.129502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:30.129530 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:30.132044 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.132679 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.132682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.132684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.132852 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.133151 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.133154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.133156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.133381 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:30.133544 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:30.133743 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 19:33:30.133889 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:33:30.134832 10786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:30.217017 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10663
I20260926 19:33:30.232966 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10797
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:33:30.248485 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:30.233864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.6.215.194:45109)
0926 19:33:30.234443 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:30.234451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:30.234476 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:33:30.234539 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:30.234546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:30.234763 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:30.234897 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:30.236227 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.237485 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.238766 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.239293 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.239389 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:30.243456 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:30.243466 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:30.243480 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:30.243510 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:30.246210 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.246836 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.246843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.246845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.246954 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.247270 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.247273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.247275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.247571 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:30.247787 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:30.248017 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 19:33:30.248193 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":310,"threads_started":1}
I20260926 19:33:30.249768 10920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:30.333007 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10797
I20260926 19:33:30.348572 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10930
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:30.365007 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:30.349556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.6.215.195:43669)
0926 19:33:30.350157 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:30.350164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:30.350191 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:33:30.350249 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:30.350255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:30.350456 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:30.350631 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:30.351875 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.352946 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.354169 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:30.354735 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:30.354874 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:30.359315 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:30.359326 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:30.359340 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:30.359390 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:30.362181 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.362837 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.362849 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.362851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.362965 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:30.363329 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:30.363336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:30.363339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:30.363676 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:30.363983 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:30.364274 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 19:33:30.364552 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":323,"threads_started":1}
I20260926 19:33:30.366278 11053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:30.448215 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10930
I20260926 19:33:30.464112 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 10570
I20260926 19:33:30.465312 10632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:30.566663 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 10570
2026-09-26T19:33:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3908 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[11077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[11077](info): set up 2 sockets
Sep 26 19:33:30 dist-test-slave-phbh krb5kdc[11077](info): commencing operation
krb5kdc: starting...
W20260926 19:33:31.731873 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:31 dist-test-slave-phbh krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:31.930037 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:38215
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:38215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:32.245563 11090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:32.245925 11090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:32.246100 11090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:32.253262 11090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:32.253419 11090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:32.253535 11090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:32.253644 11090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:32.253724 11090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:32.263942 11090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:38215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:38215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:32.265441 11090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:32.266723 11090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:32.278564 11095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:32.278486 11096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:32.278537 11098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:32.279421 11090 server_base.cc:1034] running on GCE node
I20260926 19:33:32.280212 11090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:32.281700 11090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:32.282956 11090 hybrid_clock.cc:648] HybridClock initialized: now 1790451212282905 us; error 60 us; skew 500 ppm
I20260926 19:33:32.283341 11090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:32 dist-test-slave-phbh krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451212, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:32.286664 11090 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:32.287757 11090 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:32.287842 11090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:32.288091 11090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:32.291042 11090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "18ec6849bd85498eb4446aeabce9b868"
format_stamp: "Formatted at 2026-09-26 19:33:32 on dist-test-slave-phbh"
server_key: "b8e6713ce9e5e82379dcea48ed976918"
server_key_iv: "0ecdd996652cb6fc9a9c7015671a05bf"
server_key_version: "encryptionkey@0"
I20260926 19:33:32.292145 11090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "18ec6849bd85498eb4446aeabce9b868"
format_stamp: "Formatted at 2026-09-26 19:33:32 on dist-test-slave-phbh"
server_key: "b8e6713ce9e5e82379dcea48ed976918"
server_key_iv: "0ecdd996652cb6fc9a9c7015671a05bf"
server_key_version: "encryptionkey@0"
I20260926 19:33:32.297045 11090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 19:33:32.300349 11104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:32.302080 11090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:32.302238 11090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "18ec6849bd85498eb4446aeabce9b868"
format_stamp: "Formatted at 2026-09-26 19:33:32 on dist-test-slave-phbh"
server_key: "b8e6713ce9e5e82379dcea48ed976918"
server_key_iv: "0ecdd996652cb6fc9a9c7015671a05bf"
server_key_version: "encryptionkey@0"
I20260926 19:33:32.302428 11090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:32.338718 11090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:32.343603 11090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:32.343845 11090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:32.362905 11090 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:38215
I20260926 19:33:32.362946 11155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:38215 every 8 connection(s)
I20260926 19:33:32.363899 11090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:32.367583 11156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:32.368479 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11090
I20260926 19:33:32.368752 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:32.369520 7007 external_mini_cluster.cc:1428] Setting key 92cc5b16c3cfc20953f6c062c7bd4332
I20260926 19:33:32.381996 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868: Bootstrap starting.
Sep 26 19:33:32 dist-test-slave-phbh krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:32.385047 11156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:32.386204 11156 log.cc:824] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:32.389810 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868: No bootstrap required, opened a new log
I20260926 19:33:32.394778 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:32.371687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.6.215.254:38215)
0926 19:33:32.372529 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:32.372544 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:32.372598 (+ 54us) server_negotiation.cc:407] Connection header received
0926 19:33:32.374330 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:32.374364 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:32.374909 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:32.375375 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:32.377070 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:32.379144 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:32.380547 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:32.381588 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:32.381766 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:32.386025 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:32.386035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:32.386055 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:32.386097 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:32.390179 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:32.390909 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:32.390917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:32.390925 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:32.391125 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:32.391496 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:32.391503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:32.391507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:32.392222 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:32.392527 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:32.392961 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 19:33:32.393408 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":353,"threads_started":1}
I20260926 19:33:32.399403 11156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ec6849bd85498eb4446aeabce9b868" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38215 } }
I20260926 19:33:32.399874 11156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:32.400038 11156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18ec6849bd85498eb4446aeabce9b868, State: Initialized, Role: FOLLOWER
I20260926 19:33:32.400666 11156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ec6849bd85498eb4446aeabce9b868" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38215 } }
I20260926 19:33:32.400950 11156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:32.401077 11156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:32.401248 11156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:32.403327 11156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ec6849bd85498eb4446aeabce9b868" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38215 } }
I20260926 19:33:32.403815 11156 leader_election.cc:302] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18ec6849bd85498eb4446aeabce9b868; no voters:
I20260926 19:33:32.404351 11156 leader_election.cc:288] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:32.404505 11161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:32.407493 11156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:32.409009 11161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [term 1 LEADER]: Becoming Leader. State: Replica: 18ec6849bd85498eb4446aeabce9b868, State: Running, Role: LEADER
I20260926 19:33:32.409789 11161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ec6849bd85498eb4446aeabce9b868" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38215 } }
I20260926 19:33:32.413412 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18ec6849bd85498eb4446aeabce9b868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ec6849bd85498eb4446aeabce9b868" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38215 } } }
I20260926 19:33:32.413811 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:32.414151 11163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18ec6849bd85498eb4446aeabce9b868. Latest consensus state: current_term: 1 leader_uuid: "18ec6849bd85498eb4446aeabce9b868" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18ec6849bd85498eb4446aeabce9b868" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38215 } } }
I20260926 19:33:32.414695 11163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868 [sys.catalog]: This master's current role is: LEADER
W20260926 19:33:32.420007 11175 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:33:32.420138 11175 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:33:32.420310 11176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:32.422405 11176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:32.431285 11176 catalog_manager.cc:1347] Generated new cluster ID: 6fca8544d88b43d2ba70f8945cba7a18
I20260926 19:33:32.431406 11176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:32.450213 11176 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:32.451930 11176 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:32.463464 11176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868: Generated new TSK 0
I20260926 19:33:32.464601 11176 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:32.522049 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38215
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:32.843829 11182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:32.844103 11182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:32.844273 11182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:32.851485 11182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:32.851653 11182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:32.851846 11182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:32.861599 11182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:32.862926 11182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:32.864228 11182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:32.877314 11187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:32.877279 11188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:32.877279 11190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:32.878130 11182 server_base.cc:1034] running on GCE node
I20260926 19:33:32.878656 11182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:32.879515 11182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:32.880751 11182 hybrid_clock.cc:648] HybridClock initialized: now 1790451212880702 us; error 65 us; skew 500 ppm
I20260926 19:33:32.881191 11182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:32 dist-test-slave-phbh krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451212, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:32.884125 11182 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:32.885339 11182 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:32.885473 11182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:32.885779 11182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:32.888622 11182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "29ed2c40ddb344caa8b7383d08b7c16a"
format_stamp: "Formatted at 2026-09-26 19:33:32 on dist-test-slave-phbh"
server_key: "56517c7bfc1bd21a992c2d17daae2a57"
server_key_iv: "b7a2ab9b21aed9d836dce2c8ff9d6053"
server_key_version: "encryptionkey@0"
I20260926 19:33:32.889530 11182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "29ed2c40ddb344caa8b7383d08b7c16a"
format_stamp: "Formatted at 2026-09-26 19:33:32 on dist-test-slave-phbh"
server_key: "56517c7bfc1bd21a992c2d17daae2a57"
server_key_iv: "b7a2ab9b21aed9d836dce2c8ff9d6053"
server_key_version: "encryptionkey@0"
I20260926 19:33:32.894634 11182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:32.898182 11196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:32.899416 11182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:33:32.899597 11182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "29ed2c40ddb344caa8b7383d08b7c16a"
format_stamp: "Formatted at 2026-09-26 19:33:32 on dist-test-slave-phbh"
server_key: "56517c7bfc1bd21a992c2d17daae2a57"
server_key_iv: "b7a2ab9b21aed9d836dce2c8ff9d6053"
server_key_version: "encryptionkey@0"
I20260926 19:33:32.899811 11182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:32.943732 11182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:32.947772 11182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:32.948055 11182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:32.949028 11182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:32.950327 11182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:32.950455 11182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:32.950587 11182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:32.950678 11182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:32.990335 11182 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:38127
I20260926 19:33:32.990418 11308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:38127 every 8 connection(s)
I20260926 19:33:32.991199 11182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:32.993052 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11182
I20260926 19:33:32.993294 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:32.994120 7007 external_mini_cluster.cc:1428] Setting key 7c7b5651d631f830b306073df084007d
Sep 26 19:33:33 dist-test-slave-phbh krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451212, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:33.013273 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:32.993742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:50695 (local address 127.6.215.254:38215)
0926 19:33:32.994214 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:32.994223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:32.995502 (+ 1279us) server_negotiation.cc:407] Connection header received
0926 19:33:32.998113 (+ 2611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:32.998117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:32.998179 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:32.998270 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:33.000269 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.001040 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.003167 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.003723 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.003851 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:33.007547 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:33.007552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:33.007555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:33.007585 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:33.010229 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:33.010960 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:33.010967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:33.010973 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:33.011108 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:33.011525 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:33.011531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:33.011534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:33.011798 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:33.011950 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:33.012640 (+ 690us) server_negotiation.cc:299] Negotiation successful
0926 19:33:33.012886 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":247,"threads_started":1}
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:33:33.014299 11312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:32.994456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:38215 (local address 127.6.215.193:50695)
0926 19:33:32.995335 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:32.995385 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:32.997805 (+ 2420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:32.998535 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:32.998549 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:32.999312 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:33.000095 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:33.000121 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.001195 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.001203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:33.002988 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:33.002999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.003960 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.003967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:33.004183 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:33.004906 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:33.004932 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:33.007266 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:33.010373 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:33.010402 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:33.010419 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:33.010836 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:33.011225 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:33.011230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:33.011234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:33.011413 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:33.011926 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:33.011934 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:33.012289 (+ 355us) client_negotiation.cc:769] Sending connection context
0926 19:33:33.012567 (+ 278us) client_negotiation.cc:240] Negotiation successful
0926 19:33:33.013127 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":393,"threads_started":1}
I20260926 19:33:33.015807 11309 heartbeater.cc:347] Connected to a master server at 127.6.215.254:38215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
I20260926 19:33:33.016157 11309 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:33.016894 11309 heartbeater.cc:511] Master 127.6.215.254:38215 requested a full tablet report, sending...
I20260926 19:33:33.018960 11121 ts_manager.cc:194] Registered new tserver with Master: 29ed2c40ddb344caa8b7383d08b7c16a (127.6.215.193:38127)
I20260926 19:33:33.020308 11121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:50695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:33.040005 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38215
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:33.359506 11315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:33.359781 11315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:33.359958 11315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:33.370302 11315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:33.370437 11315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:33.370613 11315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:33.386025 11315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:33.387349 11315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:33.388693 11315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:33.401865 11323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:33.401901 11321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:33.401942 11320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:33.402670 11315 server_base.cc:1034] running on GCE node
I20260926 19:33:33.403403 11315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:33.404563 11315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:33.405786 11315 hybrid_clock.cc:648] HybridClock initialized: now 1790451213405708 us; error 66 us; skew 500 ppm
I20260926 19:33:33.406150 11315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:33 dist-test-slave-phbh krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451213, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:33.414610 11315 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:33.416201 11315 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:33.416294 11315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:33.416553 11315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:33.419730 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "98fb6f04cec14537a041107483aa4ad8"
format_stamp: "Formatted at 2026-09-26 19:33:33 on dist-test-slave-phbh"
server_key: "3d8b94f25d6abf797c334a7bbd1a6f15"
server_key_iv: "c2aa2915d898bead4e05c1be677e557a"
server_key_version: "encryptionkey@0"
I20260926 19:33:33.420513 11315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "98fb6f04cec14537a041107483aa4ad8"
format_stamp: "Formatted at 2026-09-26 19:33:33 on dist-test-slave-phbh"
server_key: "3d8b94f25d6abf797c334a7bbd1a6f15"
server_key_iv: "c2aa2915d898bead4e05c1be677e557a"
server_key_version: "encryptionkey@0"
I20260926 19:33:33.425549 11315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 19:33:33.428902 11329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:33.430253 11315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:33:33.430413 11315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "98fb6f04cec14537a041107483aa4ad8"
format_stamp: "Formatted at 2026-09-26 19:33:33 on dist-test-slave-phbh"
server_key: "3d8b94f25d6abf797c334a7bbd1a6f15"
server_key_iv: "c2aa2915d898bead4e05c1be677e557a"
server_key_version: "encryptionkey@0"
I20260926 19:33:33.430657 11315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:33.449493 11315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:33.453945 11315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:33.454181 11315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:33.455065 11315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:33.456321 11315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:33.456403 11315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:33.456482 11315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:33.456538 11315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:33.488651 11315 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:40901
I20260926 19:33:33.488752 11441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:40901 every 8 connection(s)
I20260926 19:33:33.489538 11315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:33.498379 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11315
I20260926 19:33:33.498580 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:33.499099 7007 external_mini_cluster.cc:1428] Setting key 17a1bed877409553561960519730453f
Sep 26 19:33:33 dist-test-slave-phbh krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451213, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:33.510874 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:33.491897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:45631 (local address 127.6.215.254:38215)
0926 19:33:33.492126 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:33.492132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:33.493351 (+ 1219us) server_negotiation.cc:407] Connection header received
0926 19:33:33.495075 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:33.495079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:33.495160 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:33.495252 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:33.496655 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.497410 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.499533 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.499956 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.500051 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:33.504244 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:33.504250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:33.504253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:33.504285 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:33.507538 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:33.508334 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:33.508343 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:33.508350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:33.508489 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:33.508965 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:33.508973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:33.508979 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:33.509350 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:33.509497 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:33.510166 (+ 669us) server_negotiation.cc:299] Negotiation successful
0926 19:33:33.510416 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 19:33:33.511775 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:33.492415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:38215 (local address 127.6.215.194:45631)
0926 19:33:33.493183 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:33.493228 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:33.494779 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:33.495447 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:33.495455 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:33.495961 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:33.496510 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:33.496526 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.497543 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.497547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:33.499398 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:33.499413 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.500185 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.500191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:33.500372 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:33.501103 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:33.501131 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:33.503992 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:33.507680 (+ 3688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:33.507716 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:33.507736 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:33.508218 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:33.508626 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:33.508630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:33.508632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:33.508855 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:33.509473 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:33.509479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:33.509796 (+ 317us) client_negotiation.cc:769] Sending connection context
0926 19:33:33.510014 (+ 218us) client_negotiation.cc:240] Negotiation successful
0926 19:33:33.510451 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":356,"threads_started":1}
I20260926 19:33:33.513258 11442 heartbeater.cc:347] Connected to a master server at 127.6.215.254:38215
I20260926 19:33:33.513587 11442 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:33.514387 11442 heartbeater.cc:511] Master 127.6.215.254:38215 requested a full tablet report, sending...
I20260926 19:33:33.515975 11121 ts_manager.cc:194] Registered new tserver with Master: 98fb6f04cec14537a041107483aa4ad8 (127.6.215.194:40901)
I20260926 19:33:33.516757 11121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:45631
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:33.544292 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38215
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:33.859656 11447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:33.859930 11447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:33.860023 11447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:33.867008 11447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:33.867112 11447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:33.867230 11447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:33.876513 11447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38215
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:33.877797 11447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:33.879009 11447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:33.891327 11452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:33.891355 11453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:33.891878 11455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:33.892491 11447 server_base.cc:1034] running on GCE node
I20260926 19:33:33.893114 11447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:33.894016 11447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:33.895212 11447 hybrid_clock.cc:648] HybridClock initialized: now 1790451213895162 us; error 41 us; skew 500 ppm
I20260926 19:33:33.895538 11447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:33 dist-test-slave-phbh krb5kdc[11077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451213, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:33.898586 11447 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:33.899746 11447 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:33.899868 11447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:33.900192 11447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:33.903124 11447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "e3be12e337ef4c089fed1b641a5d0aa7"
format_stamp: "Formatted at 2026-09-26 19:33:33 on dist-test-slave-phbh"
server_key: "0c90d94af3a13d8d248c4ac38484f40c"
server_key_iv: "58669883e18b4838979488474af2fad6"
server_key_version: "encryptionkey@0"
I20260926 19:33:33.903980 11447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "e3be12e337ef4c089fed1b641a5d0aa7"
format_stamp: "Formatted at 2026-09-26 19:33:33 on dist-test-slave-phbh"
server_key: "0c90d94af3a13d8d248c4ac38484f40c"
server_key_iv: "58669883e18b4838979488474af2fad6"
server_key_version: "encryptionkey@0"
I20260926 19:33:33.909245 11447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260926 19:33:33.912636 11461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:33.913938 11447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:33.914110 11447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "e3be12e337ef4c089fed1b641a5d0aa7"
format_stamp: "Formatted at 2026-09-26 19:33:33 on dist-test-slave-phbh"
server_key: "0c90d94af3a13d8d248c4ac38484f40c"
server_key_iv: "58669883e18b4838979488474af2fad6"
server_key_version: "encryptionkey@0"
I20260926 19:33:33.914321 11447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:33.944161 11447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:33.948550 11447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:33.948868 11447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:33.949810 11447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:33.951102 11447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:33.951200 11447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:33.951331 11447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:33.951448 11447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:33.987314 11447 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:43401
I20260926 19:33:33.987355 11573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:43401 every 8 connection(s)
I20260926 19:33:33.988214 11447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:33.992446 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11447
I20260926 19:33:33.992671 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:33.993196 7007 external_mini_cluster.cc:1428] Setting key 26baf360d98b17a70ea660e9aeaede26
Sep 26 19:33:33 dist-test-slave-phbh krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451213, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:34.010039 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:33.990794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:33003 (local address 127.6.215.254:38215)
0926 19:33:33.991054 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:33.991064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:33.992319 (+ 1255us) server_negotiation.cc:407] Connection header received
0926 19:33:33.995140 (+ 2821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:33.995145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:33.995264 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:33.995373 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:33.997471 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.998220 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.999811 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.000177 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.000267 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.003707 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:34.003712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:34.003715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:34.003745 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:34.006650 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.007579 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.007584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.007587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.007795 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.008260 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.008265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.008268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.008600 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:34.008748 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.009489 (+ 741us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.009691 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:33:34.011053 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:33.991304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:38215 (local address 127.6.215.195:33003)
0926 19:33:33.992133 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:33.992195 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:33.994782 (+ 2587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:33.995664 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:33.995677 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:33.996588 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:33.997312 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:33.997334 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:33.998362 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:33.998366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:33.999682 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:33.999694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.000397 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.000402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.000579 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.001262 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:34.001285 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:34.003525 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:34.006770 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:34.006779 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:34.006797 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:34.007446 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:34.007892 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:34.007896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:34.007899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:34.008119 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:34.008721 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:34.008737 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:34.009128 (+ 391us) client_negotiation.cc:769] Sending connection context
0926 19:33:34.009395 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 19:33:34.009907 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":317,"threads_started":1}
I20260926 19:33:34.012367 11574 heartbeater.cc:347] Connected to a master server at 127.6.215.254:38215
I20260926 19:33:34.012681 11574 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:34.013322 11574 heartbeater.cc:511] Master 127.6.215.254:38215 requested a full tablet report, sending...
I20260926 19:33:34.014618 11121 ts_manager.cc:194] Registered new tserver with Master: e3be12e337ef4c089fed1b641a5d0aa7 (127.6.215.195:43401)
I20260926 19:33:34.015357 11121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:33003
I20260926 19:33:34.020361 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:33:34.023546 11309 heartbeater.cc:503] Master 127.6.215.254:38215 was elected leader, sending a full tablet report...
I20260926 19:33:34.036821 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.024919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54130 (local address 127.6.215.254:38215)
0926 19:33:34.025110 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.025116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.025664 (+ 548us) server_negotiation.cc:407] Connection header received
0926 19:33:34.025782 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.025786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.025845 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.025941 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:34.027290 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.027970 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.029218 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.029629 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.029724 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.031816 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:34.031825 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:34.031829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:34.031865 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:34.034291 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.034854 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.034857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.034859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.034987 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.035387 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.035394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.035400 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.035698 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:34.035954 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.036255 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.036459 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:33:34.040633 11121 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:34.047371 11121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:34.070096 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.058788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.6.215.194:40901)
0926 19:33:34.060453 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.060461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.060488 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:33:34.060600 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.060611 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.060899 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.061841 (+ 942us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:34.062057 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.063696 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.067456 (+ 3760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.068911 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.069047 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.069251 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.069586 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.069766 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1528,"thread_start_us":1431,"threads_started":1}
I20260926 19:33:34.070194 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.058584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:40901 (local address 127.0.0.1:45300)
0926 19:33:34.059556 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:34.059605 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:34.059845 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:34.061167 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:34.061176 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:34.061218 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:34.061657 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.061669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.063848 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.063853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.066394 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.066416 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.069131 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.069137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.069329 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.069380 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 19:33:34.069523 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 19:33:34.069705 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":22,"thread_start_us":1210,"threads_started":1}
I20260926 19:33:34.070238 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.059394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:38127 (local address 127.0.0.1:42036)
0926 19:33:34.061354 (+ 1960us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:34.061374 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:34.061489 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:34.062370 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:34.062378 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:34.062404 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:34.062815 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.062826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.064629 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.064640 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.067248 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.067259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.069404 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.069409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.069573 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.069607 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 19:33:34.069703 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 19:33:34.069822 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1829,"mutex_wait_us":1,"thread_start_us":1689,"threads_started":1}
I20260926 19:33:34.070932 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.060043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.6.215.193:38127)
0926 19:33:34.060929 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.060938 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.061623 (+ 685us) server_negotiation.cc:407] Connection header received
0926 19:33:34.061719 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.061730 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.061953 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.062157 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:34.062984 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.064467 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.067412 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.068911 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.069047 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.069279 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.070401 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.070566 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":569,"threads_started":1}
I20260926 19:33:34.074668 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.060594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:43401 (local address 127.0.0.1:32774)
0926 19:33:34.067199 (+ 6605us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:34.067219 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:34.067337 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:34.067977 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:34.067981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:34.068005 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:34.068362 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.068370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.070761 (+ 2391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.070766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.072453 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.072465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.073894 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.073900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.074062 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.074100 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 19:33:34.074230 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 19:33:34.074352 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6428,"thread_start_us":6293,"threads_started":1}
I20260926 19:33:34.074729 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.061206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.6.215.195:43401)
0926 19:33:34.067507 (+ 6301us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.067513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.067537 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:33:34.067608 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.067620 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.067790 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.067996 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:34.068562 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.070060 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.072615 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.073724 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.073849 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.074074 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.074266 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.074408 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6119,"thread_start_us":5970,"threads_started":1}
I20260926 19:33:34.074566 11377 tablet_service.cc:1467] Processing CreateTablet for tablet 7e229592945442f0afd218cba21447e6 (DEFAULT_TABLE table=test-table [id=0530c5fd5af54a9c9125f43f9f4088c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:34.074566 11244 tablet_service.cc:1467] Processing CreateTablet for tablet 7e229592945442f0afd218cba21447e6 (DEFAULT_TABLE table=test-table [id=0530c5fd5af54a9c9125f43f9f4088c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:34.076118 11244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e229592945442f0afd218cba21447e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:34.076639 11377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e229592945442f0afd218cba21447e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:34.078198 11509 tablet_service.cc:1467] Processing CreateTablet for tablet 7e229592945442f0afd218cba21447e6 (DEFAULT_TABLE table=test-table [id=0530c5fd5af54a9c9125f43f9f4088c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:34.079263 11509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e229592945442f0afd218cba21447e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:34.093705 11593 tablet_bootstrap.cc:492] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Bootstrap starting.
I20260926 19:33:34.096252 11592 tablet_bootstrap.cc:492] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Bootstrap starting.
I20260926 19:33:34.097664 11594 tablet_bootstrap.cc:492] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Bootstrap starting.
I20260926 19:33:34.099136 11593 tablet_bootstrap.cc:654] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:34.100034 11592 tablet_bootstrap.cc:654] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:34.100458 11593 log.cc:824] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:34.101243 11594 tablet_bootstrap.cc:654] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:34.101230 11592 log.cc:824] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:34.102411 11594 log.cc:824] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:34.103700 11593 tablet_bootstrap.cc:492] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: No bootstrap required, opened a new log
I20260926 19:33:34.103981 11593 ts_tablet_manager.cc:1397] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260926 19:33:34.104595 11592 tablet_bootstrap.cc:492] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: No bootstrap required, opened a new log
I20260926 19:33:34.104899 11592 ts_tablet_manager.cc:1397] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 19:33:34.105566 11594 tablet_bootstrap.cc:492] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: No bootstrap required, opened a new log
I20260926 19:33:34.105814 11594 ts_tablet_manager.cc:1397] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 19:33:34.112414 11593 raft_consensus.cc:348] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.112782 11593 raft_consensus.cc:374] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:34.112895 11593 raft_consensus.cc:729] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ed2c40ddb344caa8b7383d08b7c16a, State: Initialized, Role: FOLLOWER
I20260926 19:33:34.113529 11593 consensus_queue.cc:260] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.115286 11593 ts_tablet_manager.cc:1428] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260926 19:33:34.115509 11592 raft_consensus.cc:348] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.115916 11592 raft_consensus.cc:374] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:34.116009 11592 raft_consensus.cc:729] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3be12e337ef4c089fed1b641a5d0aa7, State: Initialized, Role: FOLLOWER
I20260926 19:33:34.116223 11594 raft_consensus.cc:348] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.116624 11594 raft_consensus.cc:374] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:34.116724 11594 raft_consensus.cc:729] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98fb6f04cec14537a041107483aa4ad8, State: Initialized, Role: FOLLOWER
I20260926 19:33:34.116583 11592 consensus_queue.cc:260] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.117357 11594 consensus_queue.cc:260] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.119437 11442 heartbeater.cc:503] Master 127.6.215.254:38215 was elected leader, sending a full tablet report...
I20260926 19:33:34.119477 11594 ts_tablet_manager.cc:1428] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260926 19:33:34.120193 11574 heartbeater.cc:503] Master 127.6.215.254:38215 was elected leader, sending a full tablet report...
I20260926 19:33:34.122232 11592 ts_tablet_manager.cc:1428] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260926 19:33:34.137797 11600 raft_consensus.cc:482] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:34.138151 11600 raft_consensus.cc:504] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.139752 11600 leader_election.cc:288] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3be12e337ef4c089fed1b641a5d0aa7 (127.6.215.195:43401), 29ed2c40ddb344caa8b7383d08b7c16a (127.6.215.193:38127)
I20260926 19:33:34.146518 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.140344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:33841 (local address 127.6.215.193:38127)
0926 19:33:34.140521 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.140528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.141048 (+ 520us) server_negotiation.cc:407] Connection header received
0926 19:33:34.141242 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.141247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.141322 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.141386 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:34.142082 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.142953 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.144751 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.145541 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.145719 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.145806 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.146134 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.146258 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 19:33:34.146523 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.140087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:55897 (local address 127.6.215.195:43401)
0926 19:33:34.140291 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.140299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.141130 (+ 831us) server_negotiation.cc:407] Connection header received
0926 19:33:34.141313 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.141316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.141398 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.141488 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:34.142157 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.142966 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.144746 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.145548 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.145730 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.145808 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.146169 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.146280 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:33:34.146703 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.140075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:43401 (local address 127.6.215.194:55897)
0926 19:33:34.140982 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:34.141015 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:34.141151 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:34.141519 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:34.141524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:34.141548 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:34.141985 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.141993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.143136 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.143139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.144595 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.144606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.145808 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.145814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.145993 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.146014 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 19:33:34.146132 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 19:33:34.146258 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":25,"thread_start_us":415,"threads_started":1}
I20260926 19:33:34.146867 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.139999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:38127 (local address 127.6.215.194:33841)
0926 19:33:34.140928 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:34.140944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:34.141064 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:34.141454 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:34.141461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:34.141507 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:34.141941 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.141950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.143077 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.143083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.144593 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:34.144604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.145778 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.145784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:34.145930 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.146000 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 19:33:34.146073 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 19:33:34.146226 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":32,"thread_start_us":613,"threads_started":1}
I20260926 19:33:34.147756 11529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e229592945442f0afd218cba21447e6" candidate_uuid: "98fb6f04cec14537a041107483aa4ad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" is_pre_election: true
I20260926 19:33:34.147800 11264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e229592945442f0afd218cba21447e6" candidate_uuid: "98fb6f04cec14537a041107483aa4ad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" is_pre_election: true
I20260926 19:33:34.148247 11529 raft_consensus.cc:2454] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98fb6f04cec14537a041107483aa4ad8 in term 0.
I20260926 19:33:34.148296 11264 raft_consensus.cc:2454] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98fb6f04cec14537a041107483aa4ad8 in term 0.
I20260926 19:33:34.148948 11330 leader_election.cc:302] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98fb6f04cec14537a041107483aa4ad8, e3be12e337ef4c089fed1b641a5d0aa7; no voters:
I20260926 19:33:34.149318 11600 raft_consensus.cc:2781] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:34.149461 11600 raft_consensus.cc:482] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:34.149569 11600 raft_consensus.cc:3037] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:34.151608 11600 raft_consensus.cc:504] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.152196 11600 leader_election.cc:288] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [CANDIDATE]: Term 1 election: Requested vote from peers e3be12e337ef4c089fed1b641a5d0aa7 (127.6.215.195:43401), 29ed2c40ddb344caa8b7383d08b7c16a (127.6.215.193:38127)
I20260926 19:33:34.152547 11529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e229592945442f0afd218cba21447e6" candidate_uuid: "98fb6f04cec14537a041107483aa4ad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3be12e337ef4c089fed1b641a5d0aa7"
I20260926 19:33:34.152645 11264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e229592945442f0afd218cba21447e6" candidate_uuid: "98fb6f04cec14537a041107483aa4ad8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ed2c40ddb344caa8b7383d08b7c16a"
I20260926 19:33:34.152745 11529 raft_consensus.cc:3037] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:34.152897 11264 raft_consensus.cc:3037] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:34.154894 11529 raft_consensus.cc:2454] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98fb6f04cec14537a041107483aa4ad8 in term 1.
I20260926 19:33:34.154970 11264 raft_consensus.cc:2454] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98fb6f04cec14537a041107483aa4ad8 in term 1.
I20260926 19:33:34.155292 11330 leader_election.cc:302] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98fb6f04cec14537a041107483aa4ad8, e3be12e337ef4c089fed1b641a5d0aa7; no voters:
I20260926 19:33:34.155586 11600 raft_consensus.cc:2781] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:34.156009 11600 raft_consensus.cc:686] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 1 LEADER]: Becoming Leader. State: Replica: 98fb6f04cec14537a041107483aa4ad8, State: Running, Role: LEADER
I20260926 19:33:34.156478 11600 consensus_queue.cc:237] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } }
I20260926 19:33:34.161314 11119 catalog_manager.cc:5495] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98fb6f04cec14537a041107483aa4ad8 (127.6.215.194). New cstate: current_term: 1 leader_uuid: "98fb6f04cec14537a041107483aa4ad8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98fb6f04cec14537a041107483aa4ad8" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 } health_report { overall_health: UNKNOWN } } }
I20260926 19:33:34.193728 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.188266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.6.215.194:40901)
0926 19:33:34.188494 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.188501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.188629 (+ 128us) server_negotiation.cc:407] Connection header received
0926 19:33:34.188923 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.188928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.189005 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.189172 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:34.189520 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.190237 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.191577 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.191972 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.192101 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.192582 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:34.193144 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:34.193311 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.193374 (+ 63us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.193452 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 19:33:34.202097 11529 raft_consensus.cc:1261] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 1 FOLLOWER]: Refusing update from remote peer 98fb6f04cec14537a041107483aa4ad8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:34.202097 11264 raft_consensus.cc:1261] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 1 FOLLOWER]: Refusing update from remote peer 98fb6f04cec14537a041107483aa4ad8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:34.203097 11603 consensus_queue.cc:1035] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3be12e337ef4c089fed1b641a5d0aa7" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:34.203476 11600 consensus_queue.cc:1035] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29ed2c40ddb344caa8b7383d08b7c16a" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:34.228641 11119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:34.228969 11119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:34.232431 11119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18ec6849bd85498eb4446aeabce9b868: Sending DeleteTablet for 3 replicas of tablet 7e229592945442f0afd218cba21447e6
I20260926 19:33:34.233603 11377 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e229592945442f0afd218cba21447e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:34 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45300
I20260926 19:33:34.233711 11244 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e229592945442f0afd218cba21447e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:34 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:42036
I20260926 19:33:34.233711 11509 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e229592945442f0afd218cba21447e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:34 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:32774
I20260926 19:33:34.234535 11615 tablet_replica.cc:330] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: stopping tablet replica
I20260926 19:33:34.235042 11615 raft_consensus.cc:2229] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:34.235885 11615 raft_consensus.cc:2258] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:34.236337 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11182
I20260926 19:33:34.250552 11615 ts_tablet_manager.cc:1905] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:34.253612 11616 tablet_replica.cc:330] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: stopping tablet replica
I20260926 19:33:34.253839 11617 tablet_replica.cc:330] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: stopping tablet replica
I20260926 19:33:34.254288 11617 raft_consensus.cc:2229] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:34.254303 11616 raft_consensus.cc:2229] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:34.254813 11617 raft_consensus.cc:2258] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:34.254969 11616 raft_consensus.cc:2258] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:34.256937 11616 ts_tablet_manager.cc:1905] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:34.256947 11617 ts_tablet_manager.cc:1905] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:34.258841 11615 ts_tablet_manager.cc:1918] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:34.259008 11615 log.cc:1196] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/7e229592945442f0afd218cba21447e6
I20260926 19:33:34.259545 11615 ts_tablet_manager.cc:1939] T 7e229592945442f0afd218cba21447e6 P 98fb6f04cec14537a041107483aa4ad8: Deleting consensus metadata
Sep 26 19:33:34 dist-test-slave-phbh krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:34.261981 11105 catalog_manager.cc:4841] TS 98fb6f04cec14537a041107483aa4ad8 (127.6.215.194:40901): tablet 7e229592945442f0afd218cba21447e6 (table test-table [id=0530c5fd5af54a9c9125f43f9f4088c0]) successfully deleted
I20260926 19:33:34.264014 11617 ts_tablet_manager.cc:1918] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:34.264192 11617 log.cc:1196] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/7e229592945442f0afd218cba21447e6
I20260926 19:33:34.264307 11616 ts_tablet_manager.cc:1918] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:34.264519 11616 log.cc:1196] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/7e229592945442f0afd218cba21447e6
I20260926 19:33:34.264701 11617 ts_tablet_manager.cc:1939] T 7e229592945442f0afd218cba21447e6 P e3be12e337ef4c089fed1b641a5d0aa7: Deleting consensus metadata
I20260926 19:33:34.265177 11616 ts_tablet_manager.cc:1939] T 7e229592945442f0afd218cba21447e6 P 29ed2c40ddb344caa8b7383d08b7c16a: Deleting consensus metadata
I20260926 19:33:34.266487 11105 catalog_manager.cc:4841] TS e3be12e337ef4c089fed1b641a5d0aa7 (127.6.215.195:43401): tablet 7e229592945442f0afd218cba21447e6 (table test-table [id=0530c5fd5af54a9c9125f43f9f4088c0]) successfully deleted
I20260926 19:33:34.266820 11106 catalog_manager.cc:4841] TS 29ed2c40ddb344caa8b7383d08b7c16a (127.6.215.193:38127): tablet 7e229592945442f0afd218cba21447e6 (table test-table [id=0530c5fd5af54a9c9125f43f9f4088c0]) successfully deleted
I20260926 19:33:34.268617 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.238010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.6.215.193:38127)
0926 19:33:34.238239 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.238248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.248054 (+ 9806us) server_negotiation.cc:407] Connection header received
0926 19:33:34.248608 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.248618 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.248742 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.248873 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:34.251143 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.252708 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.254729 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.255426 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.255527 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.260719 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:34.260734 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:34.260752 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:34.260822 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:34.265504 (+ 4682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.266506 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.266516 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.266520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.266695 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.267128 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.267133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.267137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.267576 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:34.267874 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.268030 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.268242 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 19:33:34.269371 11304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:34.357795 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11182
I20260926 19:33:34.382313 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11315
Sep 26 19:33:34 dist-test-slave-phbh krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:33:34.399987 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.383211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.6.215.194:40901)
0926 19:33:34.383373 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.383378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.383395 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:33:34.383536 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.383540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.383600 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.383694 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:34.385290 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.386065 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.388014 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.388459 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.388553 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.393833 (+ 5280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:34.393847 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:34.393863 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:34.393901 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:34.396974 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.397845 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.397852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.397854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.398006 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.398399 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.398405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.398410 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.398780 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:34.399132 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.399397 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.399637 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 19:33:34.400791 11437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:34.512321 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11315
I20260926 19:33:34.531702 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11447
Sep 26 19:33:34 dist-test-slave-phbh krb5kdc[11077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:34.546919 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:34.532431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32778 (local address 127.6.215.195:43401)
0926 19:33:34.532622 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:34.532631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:34.532654 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:33:34.532922 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:34.532928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:34.533003 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:34.533115 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:34.534338 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.535077 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.536310 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:34.536721 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:34.536905 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:34.541107 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:34.541132 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:34.541148 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:34.541184 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:34.544081 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.544739 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.544747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.544749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.544990 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:34.545374 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:34.545380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:34.545384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:34.545782 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:34.546060 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:34.546385 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 19:33:34.546604 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 19:33:34.547600 11569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:34.642467 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11447
I20260926 19:33:34.658599 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11090
I20260926 19:33:34.659713 11151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:34.766901 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11090
2026-09-26T19:33:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4200 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T19:33:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:34Z Disabled control of system clock
I20260926 19:33:34.832365 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42629
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 19:33:35.153105 11626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:35.153378 11626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:35.153460 11626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:35.160526 11626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:35.160629 11626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:35.160666 11626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:35.160722 11626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:35.160789 11626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:35.169826 11626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:35.171015 11626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:35.172654 11626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:35.184508 11632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:35.184549 11634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:35.184852 11631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:35.185236 11626 server_base.cc:1034] running on GCE node
I20260926 19:33:35.186005 11626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:35.187340 11626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:35.188570 11626 hybrid_clock.cc:648] HybridClock initialized: now 1790451215188528 us; error 55 us; skew 500 ppm
I20260926 19:33:35.188967 11626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:35.191218 11626 webserver.cc:460] Webserver started at http://127.6.215.254:41743/ using document root <none> and password file <none>
I20260926 19:33:35.191869 11626 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:35.192006 11626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:35.192317 11626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:35.195171 11626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "e3d6f53430844cf8b55bca110be82e4c"
format_stamp: "Formatted at 2026-09-26 19:33:35 on dist-test-slave-phbh"
server_key: "f73712b811d0274af1002ca9312df4a8"
server_key_iv: "6466894ab60f9ef18175e21405d652ae"
server_key_version: "encryptionkey@0"
I20260926 19:33:35.195943 11626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "e3d6f53430844cf8b55bca110be82e4c"
format_stamp: "Formatted at 2026-09-26 19:33:35 on dist-test-slave-phbh"
server_key: "f73712b811d0274af1002ca9312df4a8"
server_key_iv: "6466894ab60f9ef18175e21405d652ae"
server_key_version: "encryptionkey@0"
I20260926 19:33:35.200846 11626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 19:33:35.204093 11640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:35.205483 11626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:33:35.205665 11626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "e3d6f53430844cf8b55bca110be82e4c"
format_stamp: "Formatted at 2026-09-26 19:33:35 on dist-test-slave-phbh"
server_key: "f73712b811d0274af1002ca9312df4a8"
server_key_iv: "6466894ab60f9ef18175e21405d652ae"
server_key_version: "encryptionkey@0"
I20260926 19:33:35.205878 11626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:35.211429 11626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:35.212157 11626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:35.212443 11626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:35.230350 11626 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42629
I20260926 19:33:35.230367 11691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42629 every 8 connection(s)
I20260926 19:33:35.231668 11626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:35.235231 11692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:35.239766 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11626
I20260926 19:33:35.239954 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:35.240837 7007 external_mini_cluster.cc:1428] Setting key dd1d38923bfa0d60db2a06831b07de82
I20260926 19:33:35.246886 11692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c: Bootstrap starting.
I20260926 19:33:35.249305 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:35.242846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35498 (local address 127.6.215.254:42629)
0926 19:33:35.243868 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:35.243878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:35.243923 (+ 45us) server_negotiation.cc:407] Connection header received
0926 19:33:35.245677 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:35.245699 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:35.246109 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:35.246430 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:35.247578 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:35.247584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:35.247597 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:35.247611 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:35.247667 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:35.247683 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:35.247846 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:35.247995 (+ 149us) server_negotiation.cc:299] Negotiation successful
0926 19:33:35.248265 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":401,"threads_started":1}
I20260926 19:33:35.250703 11692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:35.251753 11692 log.cc:824] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:35.254662 11692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c: No bootstrap required, opened a new log
I20260926 19:33:35.262104 11692 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d6f53430844cf8b55bca110be82e4c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42629 } }
I20260926 19:33:35.262544 11692 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:35.262657 11692 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d6f53430844cf8b55bca110be82e4c, State: Initialized, Role: FOLLOWER
I20260926 19:33:35.263278 11692 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d6f53430844cf8b55bca110be82e4c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42629 } }
I20260926 19:33:35.263479 11692 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:35.263573 11692 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:35.263700 11692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:35.265774 11692 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d6f53430844cf8b55bca110be82e4c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42629 } }
I20260926 19:33:35.266265 11692 leader_election.cc:302] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3d6f53430844cf8b55bca110be82e4c; no voters:
I20260926 19:33:35.266821 11692 leader_election.cc:288] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:35.266970 11697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:35.268138 11697 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [term 1 LEADER]: Becoming Leader. State: Replica: e3d6f53430844cf8b55bca110be82e4c, State: Running, Role: LEADER
I20260926 19:33:35.268637 11697 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d6f53430844cf8b55bca110be82e4c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42629 } }
I20260926 19:33:35.269053 11692 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:35.271449 11699 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3d6f53430844cf8b55bca110be82e4c. Latest consensus state: current_term: 1 leader_uuid: "e3d6f53430844cf8b55bca110be82e4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d6f53430844cf8b55bca110be82e4c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42629 } } }
I20260926 19:33:35.271654 11698 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3d6f53430844cf8b55bca110be82e4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3d6f53430844cf8b55bca110be82e4c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42629 } } }
I20260926 19:33:35.271888 11699 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:35.271968 11698 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:35.272742 11705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:35.278362 11705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:35.286835 11705 catalog_manager.cc:1347] Generated new cluster ID: 60978557ae8d4668be540fed0faafa34
I20260926 19:33:35.286942 11705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:35.297346 11705 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:35.297761 11705 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:35.304565 11705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c: Generated new TSK 0
I20260926 19:33:35.305408 11705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:33:35.311728 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42629
--builtin_ntp_servers=127.6.215.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 19:33:35.650135 11716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:35.650414 11716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:35.650575 11716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:35.657915 11716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:35.658042 11716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:35.658229 11716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:35.667389 11716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:35.668696 11716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:35.670571 11716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:35.685273 11721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:35.685165 11722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:35.685230 11724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:35.685983 11716 server_base.cc:1034] running on GCE node
I20260926 19:33:35.686511 11716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:35.687367 11716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:35.688565 11716 hybrid_clock.cc:648] HybridClock initialized: now 1790451215688521 us; error 32 us; skew 500 ppm
I20260926 19:33:35.688905 11716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:35.691092 11716 webserver.cc:460] Webserver started at http://127.6.215.193:33467/ using document root <none> and password file <none>
I20260926 19:33:35.691692 11716 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:35.691787 11716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:35.692157 11716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:35.694944 11716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "35c82b76d31340e9a25af4d2be17170d"
format_stamp: "Formatted at 2026-09-26 19:33:35 on dist-test-slave-phbh"
server_key: "2c811d6c0fe90c497dcb8deca8373573"
server_key_iv: "2d505668f1ff35956c95676c4752918e"
server_key_version: "encryptionkey@0"
I20260926 19:33:35.695708 11716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "35c82b76d31340e9a25af4d2be17170d"
format_stamp: "Formatted at 2026-09-26 19:33:35 on dist-test-slave-phbh"
server_key: "2c811d6c0fe90c497dcb8deca8373573"
server_key_iv: "2d505668f1ff35956c95676c4752918e"
server_key_version: "encryptionkey@0"
I20260926 19:33:35.700654 11716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 19:33:35.704056 11730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:35.705497 11716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:35.705662 11716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "35c82b76d31340e9a25af4d2be17170d"
format_stamp: "Formatted at 2026-09-26 19:33:35 on dist-test-slave-phbh"
server_key: "2c811d6c0fe90c497dcb8deca8373573"
server_key_iv: "2d505668f1ff35956c95676c4752918e"
server_key_version: "encryptionkey@0"
I20260926 19:33:35.705883 11716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:35.711553 11716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:35.712297 11716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:35.712559 11716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:35.713514 11716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:35.714830 11716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:35.714941 11716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:35.715067 11716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:35.715159 11716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:35.747599 11716 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:33861
I20260926 19:33:35.747677 11842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:33861 every 8 connection(s)
I20260926 19:33:35.748982 11716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:35.753245 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11716
I20260926 19:33:35.753450 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:35.754251 7007 external_mini_cluster.cc:1428] Setting key 06ab374625c3266357e1a7c6821d1f59
I20260926 19:33:35.757280 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42629
--builtin_ntp_servers=127.6.215.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 19:33:35.757889 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:35.751546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:39629 (local address 127.6.215.254:42629)
0926 19:33:35.751945 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:35.751950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:35.753375 (+ 1425us) server_negotiation.cc:407] Connection header received
0926 19:33:35.755576 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:35.755581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:35.755634 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:35.755747 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:35.757104 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:35.757108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:35.757111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:35.757126 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:35.757165 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:35.757176 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:35.757266 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:35.757624 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 19:33:35.757678 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":211,"threads_started":1}
I20260926 19:33:35.759181 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:35.752162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42629 (local address 127.6.215.193:39629)
0926 19:33:35.753179 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:35.753232 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:35.755333 (+ 2101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:35.755926 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:35.755934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:35.756022 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:35.756722 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:35.756725 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:35.756794 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:35.756903 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:35.757283 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:35.757318 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:35.757326 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 19:33:35.757687 (+ 361us) client_negotiation.cc:240] Negotiation successful
0926 19:33:35.757958 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":242,"threads_started":1}
I20260926 19:33:35.760655 11843 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42629
I20260926 19:33:35.760977 11843 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:35.761595 11843 heartbeater.cc:511] Master 127.6.215.254:42629 requested a full tablet report, sending...
I20260926 19:33:35.763805 11657 ts_manager.cc:194] Registered new tserver with Master: 35c82b76d31340e9a25af4d2be17170d (127.6.215.193:33861)
W20260926 19:33:36.073302 11847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:36.073613 11847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:36.073772 11847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:36.081561 11847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:36.081681 11847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:36.081794 11847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:36.090898 11847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:36.092090 11847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:36.093824 11847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:36.106741 11852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:36.106835 11853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:36.107286 11847 server_base.cc:1034] running on GCE node
W20260926 19:33:36.106777 11855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:36.107995 11847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:36.109099 11847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:36.110368 11847 hybrid_clock.cc:648] HybridClock initialized: now 1790451216110322 us; error 45 us; skew 500 ppm
I20260926 19:33:36.110706 11847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:36.112929 11847 webserver.cc:460] Webserver started at http://127.6.215.194:44539/ using document root <none> and password file <none>
I20260926 19:33:36.113629 11847 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:36.113757 11847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:36.114060 11847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:36.116961 11847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "afad37b0830d40f6b6bd635640941357"
format_stamp: "Formatted at 2026-09-26 19:33:36 on dist-test-slave-phbh"
server_key: "80202e4fa4e509cdd4441879bd2b5ee8"
server_key_iv: "3ae01c96260a39602e9e1ded56f1bfca"
server_key_version: "encryptionkey@0"
I20260926 19:33:36.117738 11847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "afad37b0830d40f6b6bd635640941357"
format_stamp: "Formatted at 2026-09-26 19:33:36 on dist-test-slave-phbh"
server_key: "80202e4fa4e509cdd4441879bd2b5ee8"
server_key_iv: "3ae01c96260a39602e9e1ded56f1bfca"
server_key_version: "encryptionkey@0"
I20260926 19:33:36.122689 11847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 19:33:36.126063 11861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:36.127315 11847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:33:36.127486 11847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "afad37b0830d40f6b6bd635640941357"
format_stamp: "Formatted at 2026-09-26 19:33:36 on dist-test-slave-phbh"
server_key: "80202e4fa4e509cdd4441879bd2b5ee8"
server_key_iv: "3ae01c96260a39602e9e1ded56f1bfca"
server_key_version: "encryptionkey@0"
I20260926 19:33:36.127712 11847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:36.133231 11847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:36.133996 11847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:36.134258 11847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:36.135107 11847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:36.136547 11847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:36.136664 11847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:36.136804 11847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:36.136902 11847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260926 19:33:36.170403 11847 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:39515
I20260926 19:33:36.170449 11973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:39515 every 8 connection(s)
I20260926 19:33:36.171876 11847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:36.177098 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11847
I20260926 19:33:36.177289 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:36.177763 7007 external_mini_cluster.cc:1428] Setting key aa0a04658ecf23e7fe6e3253970174c2
I20260926 19:33:36.180742 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.174616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:52063 (local address 127.6.215.254:42629)
0926 19:33:36.174836 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.174843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.176370 (+ 1527us) server_negotiation.cc:407] Connection header received
0926 19:33:36.178164 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.178171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.178240 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.178371 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.179855 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.179862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.179866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.179886 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.179922 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.179938 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.180085 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.180320 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.180378 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 19:33:36.180918 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42629
--builtin_ntp_servers=127.6.215.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 19:33:36.181620 11976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.175282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42629 (local address 127.6.215.194:52063)
0926 19:33:36.176199 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:36.176241 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:36.177833 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:36.178635 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:36.178645 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:36.178732 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:36.179473 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:36.179476 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:36.179548 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:36.179661 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:36.180073 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:36.180085 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:36.180093 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 19:33:36.180373 (+ 280us) client_negotiation.cc:240] Negotiation successful
0926 19:33:36.180581 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":294,"threads_started":1}
I20260926 19:33:36.182904 11974 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42629
I20260926 19:33:36.183203 11974 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:36.183759 11974 heartbeater.cc:511] Master 127.6.215.254:42629 requested a full tablet report, sending...
I20260926 19:33:36.185074 11657 ts_manager.cc:194] Registered new tserver with Master: afad37b0830d40f6b6bd635640941357 (127.6.215.194:39515)
W20260926 19:33:36.504621 11977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:36.505002 11977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:36.505164 11977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:36.512868 11977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:36.512972 11977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:36.513147 11977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:36.522367 11977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:36.523654 11977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:36.525422 11977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:36.538793 11983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:36.538856 11985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:36.539091 11982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:36.539882 11977 server_base.cc:1034] running on GCE node
I20260926 19:33:36.540472 11977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:36.541453 11977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:36.542681 11977 hybrid_clock.cc:648] HybridClock initialized: now 1790451216542642 us; error 51 us; skew 500 ppm
I20260926 19:33:36.542989 11977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:36.545300 11977 webserver.cc:460] Webserver started at http://127.6.215.195:39383/ using document root <none> and password file <none>
I20260926 19:33:36.545957 11977 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:36.546089 11977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:36.546438 11977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:36.549489 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "92851f02c521473c80c5196907d5e7d8"
format_stamp: "Formatted at 2026-09-26 19:33:36 on dist-test-slave-phbh"
server_key: "cc0125f0269501cb75fa703e703380d2"
server_key_iv: "3e8935f743199515b2b3a0d30b715d3a"
server_key_version: "encryptionkey@0"
I20260926 19:33:36.550242 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "92851f02c521473c80c5196907d5e7d8"
format_stamp: "Formatted at 2026-09-26 19:33:36 on dist-test-slave-phbh"
server_key: "cc0125f0269501cb75fa703e703380d2"
server_key_iv: "3e8935f743199515b2b3a0d30b715d3a"
server_key_version: "encryptionkey@0"
I20260926 19:33:36.555248 11977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:36.558703 11991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:36.559979 11977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 19:33:36.560168 11977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "92851f02c521473c80c5196907d5e7d8"
format_stamp: "Formatted at 2026-09-26 19:33:36 on dist-test-slave-phbh"
server_key: "cc0125f0269501cb75fa703e703380d2"
server_key_iv: "3e8935f743199515b2b3a0d30b715d3a"
server_key_version: "encryptionkey@0"
I20260926 19:33:36.560405 11977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:36.565873 11977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:36.566633 11977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:36.566896 11977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:36.567756 11977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:36.569242 11977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:36.569360 11977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:36.569489 11977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:36.569574 11977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:36.602880 11977 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:37711
I20260926 19:33:36.602937 12103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:37711 every 8 connection(s)
I20260926 19:33:36.604318 11977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:36.609889 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 11977
I20260926 19:33:36.610069 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:36.610646 7007 external_mini_cluster.cc:1428] Setting key e62b0fda0cbf2be15fd05a145a19aaf8
I20260926 19:33:36.613291 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.606882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:48161 (local address 127.6.215.254:42629)
0926 19:33:36.607105 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.607111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.608448 (+ 1337us) server_negotiation.cc:407] Connection header received
0926 19:33:36.610718 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.610726 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.610801 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.610891 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.612478 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.612487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.612493 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.612515 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.612557 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.612573 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.612682 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.612937 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.612999 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:33:36.614362 12106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.607350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42629 (local address 127.6.215.195:48161)
0926 19:33:36.608292 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:36.608327 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:36.610395 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:36.611191 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:36.611209 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:36.611311 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:36.612113 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:36.612117 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:36.612169 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:36.612291 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:36.612680 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:36.612690 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:36.612699 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 19:33:36.612972 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 19:33:36.613220 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":258,"threads_started":1}
I20260926 19:33:36.615576 12104 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42629
I20260926 19:33:36.615825 12104 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:36.616353 12104 heartbeater.cc:511] Master 127.6.215.254:42629 requested a full tablet report, sending...
I20260926 19:33:36.617656 11657 ts_manager.cc:194] Registered new tserver with Master: 92851f02c521473c80c5196907d5e7d8 (127.6.215.195:37711)
I20260926 19:33:36.626585 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:33:36.633538 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.630854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49378 (local address 127.6.215.254:42629)
0926 19:33:36.631132 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.631141 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.631526 (+ 385us) server_negotiation.cc:407] Connection header received
0926 19:33:36.631726 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.631732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.631799 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.631910 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.632906 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.632910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.632913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.632928 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.632959 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.632970 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.633109 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.633221 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.633279 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260926 19:33:36.637365 11657 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:36.643909 11657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:36.662410 12116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.656041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:37711 (local address 127.0.0.1:53472)
0926 19:33:36.657864 (+ 1823us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:36.657895 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:36.658080 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:36.659860 (+ 1780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:36.659867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:36.659957 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:36.660944 (+ 987us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:36.660948 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:36.660979 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:36.661043 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:36.661615 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:36.661625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:36.661634 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 19:33:36.661795 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 19:33:36.661891 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1631,"thread_start_us":1516,"threads_started":1}
I20260926 19:33:36.662902 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.656308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.6.215.195:37711)
0926 19:33:36.657134 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.657140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.658031 (+ 891us) server_negotiation.cc:407] Connection header received
0926 19:33:36.658281 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.658291 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.658550 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.658672 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.661267 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.661277 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.661293 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.661310 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.661367 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.661388 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.661551 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.662437 (+ 886us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.662543 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":508,"threads_started":1}
I20260926 19:33:36.663540 12117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.656391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39486)
0926 19:33:36.658472 (+ 2081us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:36.658490 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:36.658746 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:36.660159 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:36.660166 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:36.660188 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:36.661601 (+ 1413us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:36.661604 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:36.661620 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:36.661659 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:36.662152 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:36.662156 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:36.662163 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 19:33:36.662958 (+ 795us) client_negotiation.cc:240] Negotiation successful
0926 19:33:36.663002 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1960,"thread_start_us":1808,"threads_started":1}
I20260926 19:33:36.664058 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.656607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.6.215.193:33861)
0926 19:33:36.657426 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.657432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.658574 (+ 1142us) server_negotiation.cc:407] Connection header received
0926 19:33:36.658898 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.658905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.659112 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.659254 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.661827 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.661837 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.661853 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.661877 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.661934 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.661953 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.662055 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.662850 (+ 795us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.662957 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":623,"threads_started":1}
I20260926 19:33:36.664463 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.658518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:39515 (local address 127.0.0.1:58104)
0926 19:33:36.661021 (+ 2503us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:36.661040 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:36.661138 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:36.661949 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:36.661953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:36.661974 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:36.663261 (+ 1287us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:36.663264 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:36.663275 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:36.663311 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:36.663873 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:36.663876 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:36.663880 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 19:33:36.664027 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 19:33:36.664073 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2389,"thread_start_us":2216,"threads_started":1}
I20260926 19:33:36.665177 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.659087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58104 (local address 127.6.215.194:39515)
0926 19:33:36.661476 (+ 2389us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.661490 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.661520 (+ 30us) server_negotiation.cc:407] Connection header received
0926 19:33:36.661590 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.661598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.661793 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.661935 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.663459 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.663467 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.663479 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.663509 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.663548 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.663562 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.663826 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.664018 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.664115 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2192,"thread_start_us":1414,"threads_started":1}
I20260926 19:33:36.667222 11778 tablet_service.cc:1467] Processing CreateTablet for tablet 950040969ebe45d599a13ca71c63fa5d (DEFAULT_TABLE table=test-table [id=1109ec07afe544379da6ed4dbb7d47ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:36.667228 12039 tablet_service.cc:1467] Processing CreateTablet for tablet 950040969ebe45d599a13ca71c63fa5d (DEFAULT_TABLE table=test-table [id=1109ec07afe544379da6ed4dbb7d47ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:36.668448 11778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950040969ebe45d599a13ca71c63fa5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:36.668653 12039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950040969ebe45d599a13ca71c63fa5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:36.668532 11909 tablet_service.cc:1467] Processing CreateTablet for tablet 950040969ebe45d599a13ca71c63fa5d (DEFAULT_TABLE table=test-table [id=1109ec07afe544379da6ed4dbb7d47ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:36.669874 11909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 950040969ebe45d599a13ca71c63fa5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:36.681664 12122 tablet_bootstrap.cc:492] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Bootstrap starting.
I20260926 19:33:36.685518 12123 tablet_bootstrap.cc:492] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Bootstrap starting.
I20260926 19:33:36.685586 12122 tablet_bootstrap.cc:654] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:36.687005 12122 log.cc:824] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:36.687139 12124 tablet_bootstrap.cc:492] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Bootstrap starting.
I20260926 19:33:36.688869 12123 tablet_bootstrap.cc:654] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:36.690071 12123 log.cc:824] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:36.690140 12124 tablet_bootstrap.cc:654] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:36.690389 12122 tablet_bootstrap.cc:492] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: No bootstrap required, opened a new log
I20260926 19:33:36.690692 12122 ts_tablet_manager.cc:1397] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 19:33:36.691259 12124 log.cc:824] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:36.694468 12123 tablet_bootstrap.cc:492] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: No bootstrap required, opened a new log
I20260926 19:33:36.694681 12124 tablet_bootstrap.cc:492] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: No bootstrap required, opened a new log
I20260926 19:33:36.694734 12123 ts_tablet_manager.cc:1397] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 19:33:36.694927 12124 ts_tablet_manager.cc:1397] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260926 19:33:36.701025 12122 raft_consensus.cc:348] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.701455 12122 raft_consensus.cc:374] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:36.701543 12122 raft_consensus.cc:729] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c82b76d31340e9a25af4d2be17170d, State: Initialized, Role: FOLLOWER
I20260926 19:33:36.702099 12124 raft_consensus.cc:348] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.702422 12124 raft_consensus.cc:374] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:36.702250 12122 consensus_queue.cc:260] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.702538 12124 raft_consensus.cc:729] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afad37b0830d40f6b6bd635640941357, State: Initialized, Role: FOLLOWER
I20260926 19:33:36.703135 12124 consensus_queue.cc:260] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.703361 11843 heartbeater.cc:503] Master 127.6.215.254:42629 was elected leader, sending a full tablet report...
I20260926 19:33:36.703974 12122 ts_tablet_manager.cc:1428] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260926 19:33:36.704178 11974 heartbeater.cc:503] Master 127.6.215.254:42629 was elected leader, sending a full tablet report...
I20260926 19:33:36.704790 12124 ts_tablet_manager.cc:1428] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260926 19:33:36.706089 12123 raft_consensus.cc:348] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.706485 12123 raft_consensus.cc:374] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:36.706607 12123 raft_consensus.cc:729] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92851f02c521473c80c5196907d5e7d8, State: Initialized, Role: FOLLOWER
I20260926 19:33:36.707162 12123 consensus_queue.cc:260] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.709398 12104 heartbeater.cc:503] Master 127.6.215.254:42629 was elected leader, sending a full tablet report...
I20260926 19:33:36.710098 12123 ts_tablet_manager.cc:1428] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260926 19:33:36.751245 11844 tablet.cc:2369] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:33:36.856105 12105 tablet.cc:2369] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:36.875977 12128 raft_consensus.cc:482] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:36.876358 12128 raft_consensus.cc:504] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.878058 12128 leader_election.cc:288] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92851f02c521473c80c5196907d5e7d8 (127.6.215.195:37711), afad37b0830d40f6b6bd635640941357 (127.6.215.194:39515)
I20260926 19:33:36.881476 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.878432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:44337 (local address 127.6.215.195:37711)
0926 19:33:36.878665 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.878671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.879057 (+ 386us) server_negotiation.cc:407] Connection header received
0926 19:33:36.879259 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.879268 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.879346 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.879459 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.880795 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.880801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.880806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.880831 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.880874 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.880891 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.881019 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.881164 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.881212 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 19:33:36.881649 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.878345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:37711 (local address 127.6.215.193:44337)
0926 19:33:36.878893 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:36.878909 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:36.879077 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:36.879511 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:36.879517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:36.879547 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:36.880473 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:36.880476 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:36.880492 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:36.880540 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:36.881019 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:36.881023 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:36.881026 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 19:33:36.881202 (+ 176us) client_negotiation.cc:240] Negotiation successful
0926 19:33:36.881252 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":294,"threads_started":1}
I20260926 19:33:36.882234 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.879166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:33863 (local address 127.6.215.194:39515)
0926 19:33:36.879341 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.879347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.879880 (+ 533us) server_negotiation.cc:407] Connection header received
0926 19:33:36.880108 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.880115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.880175 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.880248 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.881364 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.881370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.881375 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.881393 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.881426 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.881440 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.881573 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.881977 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.882023 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 19:33:36.882478 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.878985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:39515 (local address 127.6.215.193:33863)
0926 19:33:36.879772 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:36.879794 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:36.879965 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:36.880459 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:36.880464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:36.880485 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:36.881159 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:33:36.881163 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:36.881186 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:33:36.881226 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:36.881828 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:36.881833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:36.881840 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 19:33:36.881988 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 19:33:36.882039 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":715,"threads_started":1}
I20260926 19:33:36.882762 12059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950040969ebe45d599a13ca71c63fa5d" candidate_uuid: "35c82b76d31340e9a25af4d2be17170d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92851f02c521473c80c5196907d5e7d8" is_pre_election: true
I20260926 19:33:36.883301 12059 raft_consensus.cc:2454] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c82b76d31340e9a25af4d2be17170d in term 0.
I20260926 19:33:36.883455 11929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950040969ebe45d599a13ca71c63fa5d" candidate_uuid: "35c82b76d31340e9a25af4d2be17170d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afad37b0830d40f6b6bd635640941357" is_pre_election: true
I20260926 19:33:36.883949 11929 raft_consensus.cc:2454] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35c82b76d31340e9a25af4d2be17170d in term 0.
I20260926 19:33:36.883966 11732 leader_election.cc:302] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c82b76d31340e9a25af4d2be17170d, 92851f02c521473c80c5196907d5e7d8; no voters:
I20260926 19:33:36.884370 12128 raft_consensus.cc:2781] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:36.884512 12128 raft_consensus.cc:482] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:36.884620 12128 raft_consensus.cc:3037] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:36.886749 12128 raft_consensus.cc:504] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.887310 12128 leader_election.cc:288] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [CANDIDATE]: Term 1 election: Requested vote from peers 92851f02c521473c80c5196907d5e7d8 (127.6.215.195:37711), afad37b0830d40f6b6bd635640941357 (127.6.215.194:39515)
I20260926 19:33:36.887768 12059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950040969ebe45d599a13ca71c63fa5d" candidate_uuid: "35c82b76d31340e9a25af4d2be17170d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92851f02c521473c80c5196907d5e7d8"
I20260926 19:33:36.887849 11929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "950040969ebe45d599a13ca71c63fa5d" candidate_uuid: "35c82b76d31340e9a25af4d2be17170d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afad37b0830d40f6b6bd635640941357"
I20260926 19:33:36.887964 12059 raft_consensus.cc:3037] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:36.888037 11929 raft_consensus.cc:3037] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:36.890245 12059 raft_consensus.cc:2454] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c82b76d31340e9a25af4d2be17170d in term 1.
I20260926 19:33:36.890269 11929 raft_consensus.cc:2454] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35c82b76d31340e9a25af4d2be17170d in term 1.
I20260926 19:33:36.890618 11732 leader_election.cc:302] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35c82b76d31340e9a25af4d2be17170d, afad37b0830d40f6b6bd635640941357; no voters:
I20260926 19:33:36.890883 12128 raft_consensus.cc:2781] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:36.891342 12128 raft_consensus.cc:686] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 1 LEADER]: Becoming Leader. State: Replica: 35c82b76d31340e9a25af4d2be17170d, State: Running, Role: LEADER
I20260926 19:33:36.891927 12128 consensus_queue.cc:237] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } }
I20260926 19:33:36.896515 11656 catalog_manager.cc:5495] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d reported cstate change: term changed from 0 to 1, leader changed from <none> to 35c82b76d31340e9a25af4d2be17170d (127.6.215.193). New cstate: current_term: 1 leader_uuid: "35c82b76d31340e9a25af4d2be17170d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35c82b76d31340e9a25af4d2be17170d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 33861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 } health_report { overall_health: UNKNOWN } } }
W20260926 19:33:36.924171 11975 tablet.cc:2369] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:36.961889 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:36.959881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.6.215.193:33861)
0926 19:33:36.960070 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:36.960075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:36.960107 (+ 32us) server_negotiation.cc:407] Connection header received
0926 19:33:36.960238 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:36.960242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:36.960320 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:36.960404 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:36.961303 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:36.961307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:36.961310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:36.961322 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:36.961347 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:36.961358 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:36.961444 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:36.961538 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 19:33:36.961581 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:33:36.971071 12059 raft_consensus.cc:1261] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 1 FOLLOWER]: Refusing update from remote peer 35c82b76d31340e9a25af4d2be17170d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:36.971153 11929 raft_consensus.cc:1261] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 1 FOLLOWER]: Refusing update from remote peer 35c82b76d31340e9a25af4d2be17170d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:36.972008 12133 consensus_queue.cc:1035] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [LEADER]: Connected to new peer: Peer: permanent_uuid: "92851f02c521473c80c5196907d5e7d8" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 37711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:36.972381 12128 consensus_queue.cc:1035] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [LEADER]: Connected to new peer: Peer: permanent_uuid: "afad37b0830d40f6b6bd635640941357" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:36.983551 12136 mvcc.cc:205] Tried to move back new op lower bound from 7333688184699944960 to 7333688184393420800. Current Snapshot: MvccSnapshot[applied={T|T < 7333688184699944960}]
I20260926 19:33:37.000360 11657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:37.000613 11657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:37.004026 11657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e3d6f53430844cf8b55bca110be82e4c: Sending DeleteTablet for 3 replicas of tablet 950040969ebe45d599a13ca71c63fa5d
I20260926 19:33:37.005079 12039 tablet_service.cc:1514] Processing DeleteTablet for tablet 950040969ebe45d599a13ca71c63fa5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:36 UTC) from {username='slave'} at 127.0.0.1:53472
I20260926 19:33:37.005224 11778 tablet_service.cc:1514] Processing DeleteTablet for tablet 950040969ebe45d599a13ca71c63fa5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:36 UTC) from {username='slave'} at 127.0.0.1:39486
I20260926 19:33:37.006049 12146 tablet_replica.cc:330] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: stopping tablet replica
I20260926 19:33:37.006050 11909 tablet_service.cc:1514] Processing DeleteTablet for tablet 950040969ebe45d599a13ca71c63fa5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:36 UTC) from {username='slave'} at 127.0.0.1:58104
I20260926 19:33:37.006619 12146 raft_consensus.cc:2229] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:37.006754 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11716
I20260926 19:33:37.007298 12146 raft_consensus.cc:2258] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:37.008242 12147 tablet_replica.cc:330] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: stopping tablet replica
I20260926 19:33:37.008894 12147 raft_consensus.cc:2229] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:37.009570 12146 ts_tablet_manager.cc:1905] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:37.009815 12147 raft_consensus.cc:2258] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:37.012396 12147 ts_tablet_manager.cc:1905] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:37.018683 12148 tablet_replica.cc:330] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: stopping tablet replica
I20260926 19:33:37.019111 12148 raft_consensus.cc:2229] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:37.019582 12148 raft_consensus.cc:2258] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:37.021418 12119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.009149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.6.215.193:33861)
0926 19:33:37.009315 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:37.009322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:37.018745 (+ 9423us) server_negotiation.cc:407] Connection header received
0926 19:33:37.019223 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:37.019229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:37.019291 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:37.019374 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:37.020622 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:37.020628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:37.020633 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:37.020651 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:37.020684 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:37.020699 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:37.020817 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:37.021057 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 19:33:37.021114 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 19:33:37.021574 12148 ts_tablet_manager.cc:1905] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:37.022451 11838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:37.025893 12146 ts_tablet_manager.cc:1918] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:37.026057 12146 log.cc:1196] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/950040969ebe45d599a13ca71c63fa5d
I20260926 19:33:37.026669 12146 ts_tablet_manager.cc:1939] T 950040969ebe45d599a13ca71c63fa5d P 92851f02c521473c80c5196907d5e7d8: Deleting consensus metadata
I20260926 19:33:37.022120 12147 ts_tablet_manager.cc:1918] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:37.028214 12147 log.cc:1196] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/950040969ebe45d599a13ca71c63fa5d
I20260926 19:33:37.028923 11642 catalog_manager.cc:4841] TS 92851f02c521473c80c5196907d5e7d8 (127.6.215.195:37711): tablet 950040969ebe45d599a13ca71c63fa5d (table test-table [id=1109ec07afe544379da6ed4dbb7d47ab]) successfully deleted
I20260926 19:33:37.029631 12148 ts_tablet_manager.cc:1918] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:37.029788 12148 log.cc:1196] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/950040969ebe45d599a13ca71c63fa5d
I20260926 19:33:37.030215 12148 ts_tablet_manager.cc:1939] T 950040969ebe45d599a13ca71c63fa5d P afad37b0830d40f6b6bd635640941357: Deleting consensus metadata
I20260926 19:33:37.031599 11642 catalog_manager.cc:4841] TS afad37b0830d40f6b6bd635640941357 (127.6.215.194:39515): tablet 950040969ebe45d599a13ca71c63fa5d (table test-table [id=1109ec07afe544379da6ed4dbb7d47ab]) successfully deleted
I20260926 19:33:37.118050 12147 ts_tablet_manager.cc:1939] T 950040969ebe45d599a13ca71c63fa5d P 35c82b76d31340e9a25af4d2be17170d: Deleting consensus metadata
I20260926 19:33:37.118167 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11716
I20260926 19:33:37.135962 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11847
W20260926 19:33:37.135957 11644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.215.193:33861 (error 108)
I20260926 19:33:37.137909 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.137421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39508)
0926 19:33:37.137607 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:37.137712 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260926 19:33:37.138191 11644 catalog_manager.cc:4584] TS 35c82b76d31340e9a25af4d2be17170d (127.6.215.193:33861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 950040969ebe45d599a13ca71c63fa5d: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
I20260926 19:33:37.139014 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.136876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.6.215.194:39515)
0926 19:33:37.137068 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:37.137072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:37.137090 (+ 18us) server_negotiation.cc:407] Connection header received
0926 19:33:37.137243 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:37.137247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:37.137325 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:37.137407 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:37.138384 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:37.138390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:37.138394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:37.138414 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:37.138449 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:37.138464 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:37.138623 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:37.138715 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 19:33:37.138771 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:33:37.139662 11969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:37.191629 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.191201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39520)
0926 19:33:37.191399 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:37.191493 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 19:33:37.192588 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.192271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39522)
0926 19:33:37.192411 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:37.192482 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 19:33:37.226164 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11847
I20260926 19:33:37.243610 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11977
I20260926 19:33:37.246553 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.244366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.6.215.195:37711)
0926 19:33:37.244544 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:37.244551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:37.244596 (+ 45us) server_negotiation.cc:407] Connection header received
0926 19:33:37.244841 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:37.244845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:37.244919 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:37.245031 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:37.245889 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:37.245894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:37.245899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:37.245924 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:37.245954 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:37.245965 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:37.246102 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:37.246216 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 19:33:37.246276 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:33:37.247274 12099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:37.253093 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.252572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39528)
0926 19:33:37.252807 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:37.252923 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 19:33:37.254060 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.253691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39540)
0926 19:33:37.253835 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:37.253927 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 19:33:37.335031 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.334422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39550)
0926 19:33:37.334680 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:37.334812 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 19:33:37.336092 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11977
I20260926 19:33:37.336194 12120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.335772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:33861 (local address 127.0.0.1:39558)
0926 19:33:37.335964 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:37.336044 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:33861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 19:33:37.352479 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 11626
I20260926 19:33:37.353719 11687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:37.463117 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 11626
2026-09-26T19:33:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2695 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 19:33:37.505016 7007 mini_oidc.cc:152] Starting JWKS server
I20260926 19:33:37.507477 7007 webserver.cc:460] Webserver started at https://127.0.0.1:43189/ using document root <none> and password file <none>
I20260926 19:33:37.507822 7007 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 19:33:37.508453 7007 webserver.cc:460] Webserver started at http://127.0.0.1:41655/ using document root <none> and password file <none>
2026-09-26T19:33:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:37Z Disabled control of system clock
I20260926 19:33:37.535112 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37549
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:43933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:37549
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43189/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/testchainca.cert with env {}
W20260926 19:33:37.852787 12160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:37.853070 12160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:37.853155 12160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:37.860183 12160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:37.860298 12160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:37.860335 12160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:37.860383 12160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:37.860426 12160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 19:33:37.860487 12160 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 19:33:37.860527 12160 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43189/jwks.json
I20260926 19:33:37.870431 12160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:43933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:37549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43189/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:37.871809 12160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:37.873061 12160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:37.884754 12165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:37.884743 12168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:37.884785 12166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:37.885674 12160 server_base.cc:1034] running on GCE node
I20260926 19:33:37.886480 12160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:37.887849 12160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:37.889076 12160 hybrid_clock.cc:648] HybridClock initialized: now 1790451217889041 us; error 55 us; skew 500 ppm
I20260926 19:33:37.889421 12160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:37.891779 12160 webserver.cc:460] Webserver started at http://127.6.215.254:37875/ using document root <none> and password file <none>
I20260926 19:33:37.892488 12160 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:37.892627 12160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:37.892997 12160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:37.895766 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "3107e10fd6d844ae801252a45a34f788"
format_stamp: "Formatted at 2026-09-26 19:33:37 on dist-test-slave-phbh"
server_key: "9f2a1a17ca88b316399148bbf4974b34"
server_key_iv: "2e410d3f9565a3d0d044bf9223423703"
server_key_version: "encryptionkey@0"
I20260926 19:33:37.896560 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "3107e10fd6d844ae801252a45a34f788"
format_stamp: "Formatted at 2026-09-26 19:33:37 on dist-test-slave-phbh"
server_key: "9f2a1a17ca88b316399148bbf4974b34"
server_key_iv: "2e410d3f9565a3d0d044bf9223423703"
server_key_version: "encryptionkey@0"
I20260926 19:33:37.901470 12160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 19:33:37.904738 12174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:37.906168 12160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:33:37.906373 12160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "3107e10fd6d844ae801252a45a34f788"
format_stamp: "Formatted at 2026-09-26 19:33:37 on dist-test-slave-phbh"
server_key: "9f2a1a17ca88b316399148bbf4974b34"
server_key_iv: "2e410d3f9565a3d0d044bf9223423703"
server_key_version: "encryptionkey@0"
I20260926 19:33:37.906595 12160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:37.926612 12160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:37.927477 12160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:37.927798 12160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:37.946067 12160 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:37549
I20260926 19:33:37.946134 12225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:37549 every 8 connection(s)
I20260926 19:33:37.947436 12160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:37.951131 12226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:37.953169 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12160
I20260926 19:33:37.953394 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:37.954200 7007 external_mini_cluster.cc:1428] Setting key b500303de0a2993c13bb6291debd611e
I20260926 19:33:37.963155 12226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788: Bootstrap starting.
I20260926 19:33:37.966791 12226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:37.968102 12226 log.cc:824] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:37.968443 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:37.956145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38016 (local address 127.6.215.254:37549)
0926 19:33:37.957263 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:37.957274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:37.957318 (+ 44us) server_negotiation.cc:407] Connection header received
0926 19:33:37.959006 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:37.959029 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:37.959501 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:37.959907 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:37.960502 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:37.961890 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:37.963252 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:37.963988 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:37.964191 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:37.965524 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:37.965535 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:37.965552 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:37.966258 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:37.966385 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:37.966411 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:37.966535 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:37.966819 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 19:33:37.967175 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":360,"threads_started":1}
I20260926 19:33:37.973446 12226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788: No bootstrap required, opened a new log
I20260926 19:33:37.974346 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:33:37.982359 12226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3107e10fd6d844ae801252a45a34f788" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37549 } }
I20260926 19:33:37.982699 12226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:37.982775 12226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3107e10fd6d844ae801252a45a34f788, State: Initialized, Role: FOLLOWER
I20260926 19:33:37.983330 12226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3107e10fd6d844ae801252a45a34f788" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37549 } }
I20260926 19:33:37.983515 12226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:37.983609 12226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:37.983733 12226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:37.985869 12226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3107e10fd6d844ae801252a45a34f788" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37549 } }
I20260926 19:33:37.986315 12226 leader_election.cc:302] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3107e10fd6d844ae801252a45a34f788; no voters:
I20260926 19:33:37.986821 12226 leader_election.cc:288] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:37.987035 12231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:37.987768 12231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [term 1 LEADER]: Becoming Leader. State: Replica: 3107e10fd6d844ae801252a45a34f788, State: Running, Role: LEADER
I20260926 19:33:37.988335 12231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3107e10fd6d844ae801252a45a34f788" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37549 } }
I20260926 19:33:37.989212 12226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:37.991567 12233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3107e10fd6d844ae801252a45a34f788. Latest consensus state: current_term: 1 leader_uuid: "3107e10fd6d844ae801252a45a34f788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3107e10fd6d844ae801252a45a34f788" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37549 } } }
I20260926 19:33:37.991961 12233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:37.991714 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3107e10fd6d844ae801252a45a34f788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3107e10fd6d844ae801252a45a34f788" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37549 } } }
I20260926 19:33:37.992084 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:37.994077 12237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:38.000427 12237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:38.009703 12237 catalog_manager.cc:1347] Generated new cluster ID: bd1fc13677654f11a8bf9fba11aab1d8
I20260926 19:33:38.009850 12237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:38.027182 12237 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:38.028477 12237 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:38.036077 12237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3107e10fd6d844ae801252a45a34f788: Generated new TSK 0
I20260926 19:33:38.037245 12237 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 19:33:38.084579 12256 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 19:33:38.084762 12256 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 19:33:38.085512 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:38.056828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.6.215.254:37549)
0926 19:33:38.057072 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:38.057078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:38.057503 (+ 425us) server_negotiation.cc:407] Connection header received
0926 19:33:38.057844 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:38.057850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:38.057931 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:38.058053 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0926 19:33:38.058676 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.059396 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.061495 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.061947 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.062036 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:38.062666 (+ 630us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 19:33:38.085002 (+ 22336us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 19:33:38.085297 (+ 295us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38026: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 19:33:38.085680 12229 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38026: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 19:33:38.087450 12255 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:38.056676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:37549 (local address 127.0.0.1:38026)
0926 19:33:38.057383 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:38.057411 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:38.057609 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:38.058120 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:38.058125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:38.058139 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0926 19:33:38.058521 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:38.058533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.059532 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.059538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:38.061361 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:38.061376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.062104 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.062110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:38.062299 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:38.062385 (+ 86us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 19:33:38.085319 (+ 22934us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 19:33:38.085700 (+ 381us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.215.254:37549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":382,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.215.254:37549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43189/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 19:33:38.089257 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12160
I20260926 19:33:38.090461 12221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:38.183185 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12160
2026-09-26T19:33:38Z chronyd exiting
I20260926 19:33:38.232928 7007 test_util.cc:175] -----------------------------------------------
I20260926 19:33:38.233060 7007 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790451176367082-7007-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (732 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 19:33:38.236964 7007 mini_oidc.cc:152] Starting JWKS server
I20260926 19:33:38.238844 7007 webserver.cc:460] Webserver started at https://127.0.0.1:42261/ using document root <none> and password file <none>
I20260926 19:33:38.239082 7007 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 19:33:38.239708 7007 webserver.cc:460] Webserver started at http://127.0.0.1:41361/ using document root <none> and password file <none>
2026-09-26T19:33:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:38Z Disabled control of system clock
I20260926 19:33:38.266283 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42645
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:36573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42645
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42261/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/testchainca.cert with env {}
W20260926 19:33:38.602238 12263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:38.602583 12263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:38.602749 12263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:38.609975 12263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:38.610137 12263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:38.610240 12263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:38.610347 12263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:38.610435 12263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 19:33:38.610523 12263 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 19:33:38.610584 12263 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42261/jwks.json
I20260926 19:33:38.620604 12263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42261/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:38.621984 12263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:38.623293 12263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:38.634909 12268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:38.634860 12271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:38.634940 12269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:38.636121 12263 server_base.cc:1034] running on GCE node
I20260926 19:33:38.636945 12263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:38.638306 12263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:38.639536 12263 hybrid_clock.cc:648] HybridClock initialized: now 1790451218639499 us; error 26 us; skew 500 ppm
I20260926 19:33:38.639870 12263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:38.642048 12263 webserver.cc:460] Webserver started at http://127.6.215.254:37681/ using document root <none> and password file <none>
I20260926 19:33:38.642776 12263 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:38.642908 12263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:38.643234 12263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:38.646023 12263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "42380c4b53594580b5f932c678466e23"
format_stamp: "Formatted at 2026-09-26 19:33:38 on dist-test-slave-phbh"
server_key: "24504e9bd50f097c6a6b3b5983d42389"
server_key_iv: "e7afe5eaacfe2e4e0cb5fa78e3819740"
server_key_version: "encryptionkey@0"
I20260926 19:33:38.646800 12263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "42380c4b53594580b5f932c678466e23"
format_stamp: "Formatted at 2026-09-26 19:33:38 on dist-test-slave-phbh"
server_key: "24504e9bd50f097c6a6b3b5983d42389"
server_key_iv: "e7afe5eaacfe2e4e0cb5fa78e3819740"
server_key_version: "encryptionkey@0"
I20260926 19:33:38.651633 12263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 19:33:38.654934 12277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:38.656239 12263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:33:38.656415 12263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "42380c4b53594580b5f932c678466e23"
format_stamp: "Formatted at 2026-09-26 19:33:38 on dist-test-slave-phbh"
server_key: "24504e9bd50f097c6a6b3b5983d42389"
server_key_iv: "e7afe5eaacfe2e4e0cb5fa78e3819740"
server_key_version: "encryptionkey@0"
I20260926 19:33:38.656618 12263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:38.675645 12263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:38.676522 12263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:38.676888 12263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:38.694814 12263 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42645
I20260926 19:33:38.694833 12328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42645 every 8 connection(s)
I20260926 19:33:38.696135 12263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:38.699698 12329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:38.704108 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12263
I20260926 19:33:38.704306 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:38.705155 7007 external_mini_cluster.cc:1428] Setting key 0e7a64b1ff25235640411173a9fe09a3
I20260926 19:33:38.710800 12329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23: Bootstrap starting.
I20260926 19:33:38.714174 12329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:38.715358 12329 log.cc:824] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:38.718350 12329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23: No bootstrap required, opened a new log
I20260926 19:33:38.718624 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:38.706931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34952 (local address 127.6.215.254:42645)
0926 19:33:38.707904 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:38.707914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:38.707952 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:33:38.709734 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:38.709756 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:38.710177 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:38.710479 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:38.711040 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.712325 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.714184 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.714895 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.715026 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:38.716235 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:38.716245 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:38.716265 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:38.716972 (+ 707us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:38.717048 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:38.717066 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:38.717153 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:38.717372 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 19:33:38.717707 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":372,"threads_started":1}
I20260926 19:33:38.722445 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:33:38.727659 12329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42380c4b53594580b5f932c678466e23" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:33:38.727957 12329 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:38.728026 12329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42380c4b53594580b5f932c678466e23, State: Initialized, Role: FOLLOWER
I20260926 19:33:38.728541 12329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42380c4b53594580b5f932c678466e23" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:33:38.728736 12329 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:38.728909 12329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:38.729035 12329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:38.731323 12329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42380c4b53594580b5f932c678466e23" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:33:38.731745 12329 leader_election.cc:302] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42380c4b53594580b5f932c678466e23; no voters:
I20260926 19:33:38.732254 12329 leader_election.cc:288] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:38.732398 12334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:38.733139 12334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [term 1 LEADER]: Becoming Leader. State: Replica: 42380c4b53594580b5f932c678466e23, State: Running, Role: LEADER
I20260926 19:33:38.733660 12334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42380c4b53594580b5f932c678466e23" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:33:38.734206 12329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:38.736827 12336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42380c4b53594580b5f932c678466e23. Latest consensus state: current_term: 1 leader_uuid: "42380c4b53594580b5f932c678466e23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42380c4b53594580b5f932c678466e23" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } } }
I20260926 19:33:38.736914 12335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42380c4b53594580b5f932c678466e23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42380c4b53594580b5f932c678466e23" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } } }
I20260926 19:33:38.737207 12336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:38.737289 12335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:38.741761 12341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:38.747949 12341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:38.756732 12341 catalog_manager.cc:1347] Generated new cluster ID: 016ef43b8ebf427aa695f98bdfaf24e2
I20260926 19:33:38.756870 12341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:38.768632 12341 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:38.770087 12341 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:38.776589 12341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42380c4b53594580b5f932c678466e23: Generated new TSK 0
I20260926 19:33:38.777396 12341 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 19:33:38.817659 12359 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 19:33:38.817798 12359 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 19:33:38.819022 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:38.799049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34968 (local address 127.6.215.254:42645)
0926 19:33:38.799234 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:38.799241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:38.799600 (+ 359us) server_negotiation.cc:407] Connection header received
0926 19:33:38.799793 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:38.799799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:38.799894 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:38.800076 (+ 182us) server_negotiation.cc:226] Negotiated authn=JWT
0926 19:33:38.800483 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.801634 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.802967 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.803675 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.803793 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:38.804259 (+ 466us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 19:33:38.818359 (+ 14100us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 19:33:38.818731 (+ 372us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34968: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 19:33:38.819115 12358 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:38.798898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42645 (local address 127.0.0.1:34968)
0926 19:33:38.799498 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:38.799513 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:38.799634 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:38.800077 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:38.800082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:38.800093 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0926 19:33:38.800329 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:38.800336 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.801762 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.801765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:38.802833 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:38.802843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:38.803848 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:38.803852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:38.803977 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:38.804007 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 19:33:38.818616 (+ 14609us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 19:33:38.818813 (+ 197us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.215.254:42645: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":343,"threads_started":1}
W20260926 19:33:38.819145 12332 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34968: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260926 19:33:38.820399 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12263
I20260926 19:33:38.821580 12324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:38.912215 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12263
2026-09-26T19:33:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (730 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 19:33:38.967173 7007 mini_oidc.cc:152] Starting JWKS server
I20260926 19:33:38.968968 7007 webserver.cc:460] Webserver started at https://127.0.0.1:43801/ using document root <none> and password file <none>
I20260926 19:33:38.969158 7007 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 19:33:38.969708 7007 webserver.cc:460] Webserver started at http://127.0.0.1:43221/ using document root <none> and password file <none>
2026-09-26T19:33:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:38Z Disabled control of system clock
I20260926 19:33:38.997004 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42661
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:36919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42661
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43801/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 19:33:39.313197 12366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:39.313481 12366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:39.313659 12366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:39.320715 12366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:39.320847 12366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:39.320950 12366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:39.321055 12366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:39.321120 12366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 19:33:39.321228 12366 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 19:33:39.321280 12366 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43801/jwks.json
I20260926 19:33:39.330302 12366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43801/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:39.331640 12366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:39.333493 12366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:39.345382 12374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:39.345454 12372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:39.345461 12371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:39.346149 12366 server_base.cc:1034] running on GCE node
I20260926 19:33:39.346920 12366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:39.348340 12366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:39.349575 12366 hybrid_clock.cc:648] HybridClock initialized: now 1790451219349530 us; error 59 us; skew 500 ppm
I20260926 19:33:39.349936 12366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:39.352288 12366 webserver.cc:460] Webserver started at http://127.6.215.254:40943/ using document root <none> and password file <none>
I20260926 19:33:39.353098 12366 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:39.353227 12366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:39.353626 12366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:39.356643 12366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "70f53cbf12a9469d87f56edb27fa4e0c"
format_stamp: "Formatted at 2026-09-26 19:33:39 on dist-test-slave-phbh"
server_key: "557ff99ecf58da64707a30224bf46a87"
server_key_iv: "2adb1a9d59c1bc0082de88770caba835"
server_key_version: "encryptionkey@0"
I20260926 19:33:39.357511 12366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "70f53cbf12a9469d87f56edb27fa4e0c"
format_stamp: "Formatted at 2026-09-26 19:33:39 on dist-test-slave-phbh"
server_key: "557ff99ecf58da64707a30224bf46a87"
server_key_iv: "2adb1a9d59c1bc0082de88770caba835"
server_key_version: "encryptionkey@0"
I20260926 19:33:39.363179 12366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:39.366627 12380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:39.368018 12366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:33:39.368235 12366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "70f53cbf12a9469d87f56edb27fa4e0c"
format_stamp: "Formatted at 2026-09-26 19:33:39 on dist-test-slave-phbh"
server_key: "557ff99ecf58da64707a30224bf46a87"
server_key_iv: "2adb1a9d59c1bc0082de88770caba835"
server_key_version: "encryptionkey@0"
I20260926 19:33:39.368475 12366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:39.398381 12366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:39.399266 12366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:39.399592 12366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:39.422358 12366 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42661
I20260926 19:33:39.422380 12431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42661 every 8 connection(s)
I20260926 19:33:39.423764 12366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:39.424433 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12366
I20260926 19:33:39.424665 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:39.425520 7007 external_mini_cluster.cc:1428] Setting key 7f55d3b4e572f04e5a501a0861de40ad
I20260926 19:33:39.428376 12432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:39.439014 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:39.427450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.6.215.254:42661)
0926 19:33:39.428422 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:39.428434 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:39.428472 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:33:39.430126 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:39.430146 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:39.430589 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:39.430919 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:39.431498 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:39.432852 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:39.434650 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:39.435286 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:39.435410 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:39.436673 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:39.436686 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:39.436699 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:39.437310 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:39.437380 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:39.437399 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:39.437489 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:39.437723 (+ 234us) server_negotiation.cc:299] Negotiation successful
0926 19:33:39.438085 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":357,"threads_started":1}
I20260926 19:33:39.440660 12432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c: Bootstrap starting.
I20260926 19:33:39.442797 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:33:39.443955 12432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:39.445043 12432 log.cc:824] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:39.447849 12432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c: No bootstrap required, opened a new log
I20260926 19:33:39.454981 12432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42661 } }
I20260926 19:33:39.455372 12432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:39.455483 12432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f53cbf12a9469d87f56edb27fa4e0c, State: Initialized, Role: FOLLOWER
I20260926 19:33:39.456141 12432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42661 } }
I20260926 19:33:39.456416 12432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:39.456554 12432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:39.456740 12432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:39.458879 12432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42661 } }
I20260926 19:33:39.459389 12432 leader_election.cc:302] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70f53cbf12a9469d87f56edb27fa4e0c; no voters:
I20260926 19:33:39.459944 12432 leader_election.cc:288] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:39.460083 12437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:39.461205 12437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [term 1 LEADER]: Becoming Leader. State: Replica: 70f53cbf12a9469d87f56edb27fa4e0c, State: Running, Role: LEADER
I20260926 19:33:39.461848 12437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42661 } }
I20260926 19:33:39.462247 12432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:39.464809 12439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70f53cbf12a9469d87f56edb27fa4e0c. Latest consensus state: current_term: 1 leader_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42661 } } }
I20260926 19:33:39.464818 12438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70f53cbf12a9469d87f56edb27fa4e0c" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42661 } } }
I20260926 19:33:39.465631 12438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:39.465919 12439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:39.467216 12445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:39.472702 12445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:39.482580 12445 catalog_manager.cc:1347] Generated new cluster ID: 6f4d584973804a10aeb802c12863fff3
I20260926 19:33:39.482721 12445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:39.497247 12445 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:39.499034 12445 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:39.521132 12445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70f53cbf12a9469d87f56edb27fa4e0c: Generated new TSK 0
I20260926 19:33:39.522022 12445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:33:39.551824 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:39.543465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.6.215.254:42661)
0926 19:33:39.543668 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:39.543674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:39.544109 (+ 435us) server_negotiation.cc:407] Connection header received
0926 19:33:39.544301 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:39.544307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:39.544391 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:39.544548 (+ 157us) server_negotiation.cc:226] Negotiated authn=JWT
0926 19:33:39.545011 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:39.546170 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:39.547575 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:39.548258 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:39.548381 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:39.548886 (+ 505us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 19:33:39.551204 (+ 2318us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 19:33:39.551516 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51704: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 19:33:39.551952 12435 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51704: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 19:33:39.553571 12461 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:39.543334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42661 (local address 127.0.0.1:51704)
0926 19:33:39.544006 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:39.544022 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:39.544138 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:39.544572 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:39.544576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:39.544585 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 19:33:39.544875 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:39.544884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:39.546346 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:39.546350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:39.547434 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:39.547444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:39.548439 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:39.548443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:39.548568 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:39.548598 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 19:33:39.551400 (+ 2802us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 19:33:39.553142 (+ 1742us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.215.254:42661: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43801/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":1428,"thread_start_us":367,"threads_started":1}
I20260926 19:33:39.554502 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12366
I20260926 19:33:39.555733 12427 generic_service.cc:196] Checking for leaks (request via RPC)
E20260926 19:33:39.567888 12462 webserver.cc:533] Webserver: sslize failed:
E20260926 19:33:39.568040 12462 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 19:33:39.650290 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12366
2026-09-26T19:33:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (737 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T19:33:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:39Z Disabled control of system clock
I20260926 19:33:39.729928 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:34073
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:34073
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/testchainca.cert with env {}
W20260926 19:33:40.050079 12467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:40.050387 12467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:40.050534 12467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:40.058000 12467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:40.058112 12467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:40.058211 12467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:40.058322 12467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:40.058382 12467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 19:33:40.058491 12467 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 19:33:40.058545 12467 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 19:33:40.068440 12467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:34073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:34073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:40.069798 12467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:40.071107 12467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:40.083264 12473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:40.083264 12472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:40.083300 12475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:40.084180 12467 server_base.cc:1034] running on GCE node
I20260926 19:33:40.085065 12467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:40.086477 12467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:40.087668 12467 hybrid_clock.cc:648] HybridClock initialized: now 1790451220087612 us; error 58 us; skew 500 ppm
I20260926 19:33:40.088006 12467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:33:40.090277 12467 webserver.cc:460] Webserver started at http://127.6.215.254:42531/ using document root <none> and password file <none>
I20260926 19:33:40.090969 12467 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:40.091100 12467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:40.091420 12467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:40.094277 12467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "d5a6ee58398049f1bd91ed06917e4bb8"
format_stamp: "Formatted at 2026-09-26 19:33:40 on dist-test-slave-phbh"
server_key: "3c532a77d5f43228289e0d0d831eba74"
server_key_iv: "3eb570ae03ef735277cad43d7f966f4c"
server_key_version: "encryptionkey@0"
I20260926 19:33:40.095088 12467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "d5a6ee58398049f1bd91ed06917e4bb8"
format_stamp: "Formatted at 2026-09-26 19:33:40 on dist-test-slave-phbh"
server_key: "3c532a77d5f43228289e0d0d831eba74"
server_key_iv: "3eb570ae03ef735277cad43d7f966f4c"
server_key_version: "encryptionkey@0"
I20260926 19:33:40.100595 12467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260926 19:33:40.104033 12481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:40.105458 12467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:40.105656 12467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "d5a6ee58398049f1bd91ed06917e4bb8"
format_stamp: "Formatted at 2026-09-26 19:33:40 on dist-test-slave-phbh"
server_key: "3c532a77d5f43228289e0d0d831eba74"
server_key_iv: "3eb570ae03ef735277cad43d7f966f4c"
server_key_version: "encryptionkey@0"
I20260926 19:33:40.105877 12467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:40.125998 12467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:40.126835 12467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:40.127128 12467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:40.145552 12467 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:34073
I20260926 19:33:40.145570 12532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:34073 every 8 connection(s)
I20260926 19:33:40.146916 12467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:40.147884 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12467
I20260926 19:33:40.148093 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:40.148677 7007 external_mini_cluster.cc:1428] Setting key 1679005dffde180202b42727a934905e
I20260926 19:33:40.151798 12533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:40.162030 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:40.150386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.6.215.254:34073)
0926 19:33:40.151402 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:40.151417 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:40.151467 (+ 50us) server_negotiation.cc:407] Connection header received
0926 19:33:40.153120 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:40.153142 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:40.153592 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:40.153907 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:40.154478 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:40.155789 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:40.157623 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:40.158275 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:40.158418 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:40.159632 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:40.159649 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:40.159662 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:33:40.160249 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:40.160325 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:33:40.160346 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:40.160470 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:40.160681 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 19:33:40.161072 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":358,"threads_started":1}
I20260926 19:33:40.163820 12533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8: Bootstrap starting.
I20260926 19:33:40.165874 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:33:40.167083 12533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:40.168107 12533 log.cc:824] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:40.171115 12533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8: No bootstrap required, opened a new log
I20260926 19:33:40.178431 12533 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 34073 } }
I20260926 19:33:40.178788 12533 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:40.178908 12533 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5a6ee58398049f1bd91ed06917e4bb8, State: Initialized, Role: FOLLOWER
I20260926 19:33:40.179510 12533 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 34073 } }
I20260926 19:33:40.179716 12533 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:40.179845 12533 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:40.179999 12533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:40.182202 12533 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 34073 } }
I20260926 19:33:40.182684 12533 leader_election.cc:302] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5a6ee58398049f1bd91ed06917e4bb8; no voters:
I20260926 19:33:40.183246 12533 leader_election.cc:288] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:40.183403 12538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:40.184122 12538 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [term 1 LEADER]: Becoming Leader. State: Replica: d5a6ee58398049f1bd91ed06917e4bb8, State: Running, Role: LEADER
I20260926 19:33:40.184592 12538 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 34073 } }
I20260926 19:33:40.185424 12533 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:40.187625 12539 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 34073 } } }
I20260926 19:33:40.188230 12540 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5a6ee58398049f1bd91ed06917e4bb8. Latest consensus state: current_term: 1 leader_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5a6ee58398049f1bd91ed06917e4bb8" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 34073 } } }
I20260926 19:33:40.188584 12539 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:40.188680 12540 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:40.192233 12549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:40.196722 12549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:40.204722 12549 catalog_manager.cc:1347] Generated new cluster ID: 2f5a7e595c734a70b648fda9a2f49e75
I20260926 19:33:40.204860 12549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:40.228751 12549 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:40.230190 12549 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:40.237286 12549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5a6ee58398049f1bd91ed06917e4bb8: Generated new TSK 0
I20260926 19:33:40.238109 12549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:33:40.296219 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:40.253292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.6.215.254:34073)
0926 19:33:40.253476 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:40.253483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:40.253831 (+ 348us) server_negotiation.cc:407] Connection header received
0926 19:33:40.254089 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:40.254093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:40.254173 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:40.254335 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0926 19:33:40.254821 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:40.255540 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:40.256992 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:40.257697 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:40.257836 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:40.258297 (+ 461us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 19:33:40.295455 (+ 37158us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 19:33:40.295878 (+ 423us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49246: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 19:33:40.296370 12536 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49246: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 19:33:40.296375 12562 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:40.253181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:34073 (local address 127.0.0.1:49246)
0926 19:33:40.253731 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:40.253746 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:40.253905 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:40.254347 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:40.254352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:40.254364 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0926 19:33:40.254654 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:40.254664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:40.255690 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:40.255701 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:40.256844 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:40.256859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:40.257867 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:40.257871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:40.258019 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:40.258051 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 19:33:40.295791 (+ 37740us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 19:33:40.296047 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.215.254:34073: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":309,"threads_started":1}
I20260926 19:33:40.298203 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12467
I20260926 19:33:40.299417 12528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:40.397447 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12467
2026-09-26T19:33:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (726 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (403 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (401 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:41 dist-test-slave-phbh krb5kdc[12577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:41 dist-test-slave-phbh krb5kdc[12577](info): set up 2 sockets
Sep 26 19:33:41 dist-test-slave-phbh krb5kdc[12577](info): commencing operation
krb5kdc: starting...
W20260926 19:33:42.281873 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:42 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:33:42.492630 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:41623
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:41623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:42.812191 12590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:42.812552 12590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:42.812707 12590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:42.820019 12590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:42.820175 12590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:42.820281 12590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:42.820381 12590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:42.820464 12590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:42.830421 12590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:41623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:41623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:42.831773 12590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:42.833127 12590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:42.845388 12595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:42.845422 12596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:42.845592 12598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:42.846805 12590 server_base.cc:1034] running on GCE node
I20260926 19:33:42.847594 12590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:42.849045 12590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:42.850240 12590 hybrid_clock.cc:648] HybridClock initialized: now 1790451222850179 us; error 52 us; skew 500 ppm
I20260926 19:33:42.850584 12590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:42 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451222, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:42.853549 12590 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:33:42.854885 12590 webserver.cc:460] Webserver started at http://127.6.215.254:39611/ using document root <none> and password file <none>
I20260926 19:33:42.855499 12590 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:42.855603 12590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:42.855847 12590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:42.858716 12590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "1e79e494210b4c3ca20b6eafbe90a16e"
format_stamp: "Formatted at 2026-09-26 19:33:42 on dist-test-slave-phbh"
server_key: "df4ac4f97268e26bdd72fb7f5af76148"
server_key_iv: "2adf58c4bb9a363c0a1907703b405d39"
server_key_version: "encryptionkey@0"
I20260926 19:33:42.859401 12590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "1e79e494210b4c3ca20b6eafbe90a16e"
format_stamp: "Formatted at 2026-09-26 19:33:42 on dist-test-slave-phbh"
server_key: "df4ac4f97268e26bdd72fb7f5af76148"
server_key_iv: "2adf58c4bb9a363c0a1907703b405d39"
server_key_version: "encryptionkey@0"
I20260926 19:33:42.864519 12590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:33:42.867969 12605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:42.869343 12590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:42.869508 12590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "1e79e494210b4c3ca20b6eafbe90a16e"
format_stamp: "Formatted at 2026-09-26 19:33:42 on dist-test-slave-phbh"
server_key: "df4ac4f97268e26bdd72fb7f5af76148"
server_key_iv: "2adf58c4bb9a363c0a1907703b405d39"
server_key_version: "encryptionkey@0"
I20260926 19:33:42.869707 12590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:42.883507 12590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:42.887890 12590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:42.888236 12590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:42.908569 12590 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:41623
I20260926 19:33:42.908588 12656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:41623 every 8 connection(s)
I20260926 19:33:42.910112 12590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:42.912442 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12590
I20260926 19:33:42.912658 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:42.913465 7007 external_mini_cluster.cc:1428] Setting key f560eed35842c841f758d15570dd4b62
I20260926 19:33:42.914685 12657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:42.928758 12657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e: Bootstrap starting.
Sep 26 19:33:42 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:42.932850 12657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:42.934211 12657 log.cc:824] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:42.937148 12657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e: No bootstrap required, opened a new log
I20260926 19:33:42.938243 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:42.915707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54058 (local address 127.6.215.254:41623)
0926 19:33:42.916541 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:42.916554 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:42.916596 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:33:42.919175 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:42.919197 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:42.919585 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:42.919910 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:42.921700 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:42.922948 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:42.924679 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:42.925460 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:42.925617 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:42.930267 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:42.930282 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:42.930307 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:42.930365 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:42.934254 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:42.934950 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:42.934958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:42.934967 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:42.935140 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:42.935533 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:42.935562 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:42.935564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:42.936065 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:42.936355 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:42.936816 (+ 461us) server_negotiation.cc:299] Negotiation successful
0926 19:33:42.937258 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":387,"threads_started":1}
I20260926 19:33:42.945169 12657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41623 } }
I20260926 19:33:42.945544 12657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:42.945667 12657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e79e494210b4c3ca20b6eafbe90a16e, State: Initialized, Role: FOLLOWER
I20260926 19:33:42.946244 12657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41623 } }
I20260926 19:33:42.946491 12657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:42.946635 12657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:42.946799 12657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:42.949229 12657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41623 } }
I20260926 19:33:42.949806 12657 leader_election.cc:302] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e79e494210b4c3ca20b6eafbe90a16e; no voters:
I20260926 19:33:42.950343 12657 leader_election.cc:288] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:42.950484 12662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:42.951280 12662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [term 1 LEADER]: Becoming Leader. State: Replica: 1e79e494210b4c3ca20b6eafbe90a16e, State: Running, Role: LEADER
I20260926 19:33:42.951829 12662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41623 } }
I20260926 19:33:42.952500 12657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:42.954857 12663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41623 } } }
I20260926 19:33:42.955295 12663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:42.956939 12664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e79e494210b4c3ca20b6eafbe90a16e. Latest consensus state: current_term: 1 leader_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e79e494210b4c3ca20b6eafbe90a16e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41623 } } }
I20260926 19:33:42.957206 12664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:42.957742 12667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:42.963296 12667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:42.973456 12667 catalog_manager.cc:1347] Generated new cluster ID: be84ad3f858641748c40d7cdd29b2f8a
I20260926 19:33:42.973619 12667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:42.998509 12667 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:42.999804 12667 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:43.007961 12667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e79e494210b4c3ca20b6eafbe90a16e: Generated new TSK 0
I20260926 19:33:43.008857 12667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:33:43.064572 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41623
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:43.428921 12683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:43.429205 12683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:43.429401 12683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:43.436740 12683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:43.436949 12683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:43.437183 12683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:43.447662 12683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:43.449092 12683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:43.450392 12683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:43.463814 12688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:43.463764 12691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:43.463841 12689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:43.464645 12683 server_base.cc:1034] running on GCE node
I20260926 19:33:43.465308 12683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:43.466285 12683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:43.467530 12683 hybrid_clock.cc:648] HybridClock initialized: now 1790451223467484 us; error 39 us; skew 500 ppm
I20260926 19:33:43.467856 12683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:43 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451223, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:43.471097 12683 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:33:43.472642 12683 webserver.cc:460] Webserver started at http://127.6.215.193:42959/ using document root <none> and password file <none>
I20260926 19:33:43.473379 12683 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:43.473547 12683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:43.473847 12683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:43.476855 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "916b54a45f3b4344a45eb4d2f57ba4a1"
format_stamp: "Formatted at 2026-09-26 19:33:43 on dist-test-slave-phbh"
server_key: "7830503e112400a01a17d33599782ce3"
server_key_iv: "b14034e283cb0b15892b0149901fe65a"
server_key_version: "encryptionkey@0"
I20260926 19:33:43.477697 12683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "916b54a45f3b4344a45eb4d2f57ba4a1"
format_stamp: "Formatted at 2026-09-26 19:33:43 on dist-test-slave-phbh"
server_key: "7830503e112400a01a17d33599782ce3"
server_key_iv: "b14034e283cb0b15892b0149901fe65a"
server_key_version: "encryptionkey@0"
I20260926 19:33:43.482654 12683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:43.486429 12698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:43.487834 12683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:33:43.488013 12683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "916b54a45f3b4344a45eb4d2f57ba4a1"
format_stamp: "Formatted at 2026-09-26 19:33:43 on dist-test-slave-phbh"
server_key: "7830503e112400a01a17d33599782ce3"
server_key_iv: "b14034e283cb0b15892b0149901fe65a"
server_key_version: "encryptionkey@0"
I20260926 19:33:43.488214 12683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:43.506418 12683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:43.510939 12683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:43.511235 12683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:43.512149 12683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:43.513515 12683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:43.513615 12683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:43.513758 12683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:43.513832 12683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:43.547140 12683 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:39751
I20260926 19:33:43.547191 12810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:39751 every 8 connection(s)
I20260926 19:33:43.548424 12683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:43.558849 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12683
I20260926 19:33:43.559056 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:43.559824 7007 external_mini_cluster.cc:1428] Setting key 521a7a143b0e2a8a303df91fb35206c9
Sep 26 19:33:43 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451223, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:43.573216 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:43.551004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:57013 (local address 127.6.215.254:41623)
0926 19:33:43.551429 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:43.551433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:43.552472 (+ 1039us) server_negotiation.cc:407] Connection header received
0926 19:33:43.554325 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:43.554331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:43.554423 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:43.554541 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:43.555952 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:43.556658 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:43.558196 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:43.558598 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:43.558734 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:43.566366 (+ 7632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:43.566371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:43.566374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:43.566405 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:43.569748 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:43.570506 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:43.570514 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:43.570521 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:43.570661 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:43.571211 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:43.571219 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:43.571225 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:43.571582 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:43.571729 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:43.572464 (+ 735us) server_negotiation.cc:299] Negotiation successful
0926 19:33:43.572710 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":212,"threads_started":1}
I20260926 19:33:43.574256 12814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:43.551562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41623 (local address 127.6.215.193:57013)
0926 19:33:43.552310 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:43.552353 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:43.554073 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:43.554773 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:43.554781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:43.555260 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:43.555787 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:43.555802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:43.556838 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:43.556842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:43.558052 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:43.558062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:43.558819 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:43.558827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:43.559120 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:43.563329 (+ 4209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:43.563357 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:43.566131 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:43.569908 (+ 3777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:43.569917 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:43.569930 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:43.570349 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:43.570818 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:43.570822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:43.570824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:43.571071 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:43.571734 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:43.571740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:43.572074 (+ 334us) client_negotiation.cc:769] Sending connection context
0926 19:33:43.572307 (+ 233us) client_negotiation.cc:240] Negotiation successful
0926 19:33:43.572952 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":279,"threads_started":1}
I20260926 19:33:43.575912 12811 heartbeater.cc:347] Connected to a master server at 127.6.215.254:41623
I20260926 19:33:43.576215 12811 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:43.577085 12811 heartbeater.cc:511] Master 127.6.215.254:41623 requested a full tablet report, sending...
I20260926 19:33:43.579310 12622 ts_manager.cc:194] Registered new tserver with Master: 916b54a45f3b4344a45eb4d2f57ba4a1 (127.6.215.193:39751)
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:33:43.580911 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:57013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:33:43.608641 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41623
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:43.928304 12817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:43.928566 12817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:43.928659 12817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:43.935813 12817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:43.935930 12817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:43.936043 12817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:43.945817 12817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:43.947019 12817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:43.948257 12817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:43.961850 12823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:43.961875 12825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:43.962077 12822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:43.962703 12817 server_base.cc:1034] running on GCE node
I20260926 19:33:43.963312 12817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:43.964344 12817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:43.965571 12817 hybrid_clock.cc:648] HybridClock initialized: now 1790451223965526 us; error 40 us; skew 500 ppm
I20260926 19:33:43.965981 12817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:43 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451223, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:43.969173 12817 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:33:43.970578 12817 webserver.cc:460] Webserver started at http://127.6.215.194:39555/ using document root <none> and password file <none>
I20260926 19:33:43.971261 12817 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:43.971403 12817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:43.971711 12817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:43.974638 12817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "19cb382ac0804e579dcb6e82ec6c8000"
format_stamp: "Formatted at 2026-09-26 19:33:43 on dist-test-slave-phbh"
server_key: "1fc43c992c9f8e108d53f2a9532a199f"
server_key_iv: "9f856323763df5761addd2e50d61cb04"
server_key_version: "encryptionkey@0"
I20260926 19:33:43.975412 12817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "19cb382ac0804e579dcb6e82ec6c8000"
format_stamp: "Formatted at 2026-09-26 19:33:43 on dist-test-slave-phbh"
server_key: "1fc43c992c9f8e108d53f2a9532a199f"
server_key_iv: "9f856323763df5761addd2e50d61cb04"
server_key_version: "encryptionkey@0"
I20260926 19:33:43.980758 12817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 19:33:43.984210 12832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:43.985553 12817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:43.985723 12817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "19cb382ac0804e579dcb6e82ec6c8000"
format_stamp: "Formatted at 2026-09-26 19:33:43 on dist-test-slave-phbh"
server_key: "1fc43c992c9f8e108d53f2a9532a199f"
server_key_iv: "9f856323763df5761addd2e50d61cb04"
server_key_version: "encryptionkey@0"
I20260926 19:33:43.985927 12817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:44.012415 12817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:44.016700 12817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:44.017021 12817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:44.018045 12817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:44.019407 12817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:44.019512 12817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:44.019662 12817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:44.019733 12817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:44.055081 12817 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:34683
I20260926 19:33:44.055135 12944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:34683 every 8 connection(s)
I20260926 19:33:44.056361 12817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:44.058934 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12817
I20260926 19:33:44.059113 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:44.059568 7007 external_mini_cluster.cc:1428] Setting key 35ee16b306b5a43aa779d883790033b5
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451223, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:44.078042 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.058908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:33895 (local address 127.6.215.254:41623)
0926 19:33:44.059125 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.059132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.060367 (+ 1235us) server_negotiation.cc:407] Connection header received
0926 19:33:44.062222 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.062230 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.062318 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.062438 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.063894 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.065057 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.066891 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.067545 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.067646 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.071726 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.071731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.071734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.071767 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.074638 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.075616 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.075620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.075622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.075741 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.076250 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.076257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.076263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.076524 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.076643 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.077626 (+ 983us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.077792 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:33:44.079733 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.059403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41623 (local address 127.6.215.194:33895)
0926 19:33:44.060208 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:44.060253 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:44.061957 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:44.062647 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:44.062656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:44.063153 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:44.063723 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:44.063740 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.065209 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.065212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:44.066756 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:44.066767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.067784 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.067794 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:44.068126 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.068985 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:44.069008 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:44.071474 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:44.074791 (+ 3317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:44.074803 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:44.074824 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:44.075482 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:44.075853 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:44.075861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:44.075868 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:44.076126 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:44.076672 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:44.076682 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:44.077198 (+ 516us) client_negotiation.cc:769] Sending connection context
0926 19:33:44.077566 (+ 368us) client_negotiation.cc:240] Negotiation successful
0926 19:33:44.078217 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":342,"threads_started":1}
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:33:44.081267 12945 heartbeater.cc:347] Connected to a master server at 127.6.215.254:41623
I20260926 19:33:44.081583 12945 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:44.082293 12945 heartbeater.cc:511] Master 127.6.215.254:41623 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
I20260926 19:33:44.083742 12622 ts_manager.cc:194] Registered new tserver with Master: 19cb382ac0804e579dcb6e82ec6c8000 (127.6.215.194:34683)
I20260926 19:33:44.084551 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:33895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:33:44.108916 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41623
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:44.427867 12950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:44.428143 12950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:44.428303 12950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:44.435572 12950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:44.435689 12950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:44.435863 12950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:44.445950 12950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:44.447357 12950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:44.448673 12950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:44.461956 12955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:44.461828 12956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:44.461979 12958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:44.462908 12950 server_base.cc:1034] running on GCE node
I20260926 19:33:44.463526 12950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:44.464440 12950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:44.465668 12950 hybrid_clock.cc:648] HybridClock initialized: now 1790451224465629 us; error 64 us; skew 500 ppm
I20260926 19:33:44.465996 12950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.469167 12950 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:33:44.470558 12950 webserver.cc:460] Webserver started at http://127.6.215.195:44413/ using document root <none> and password file <none>
I20260926 19:33:44.471218 12950 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:44.471365 12950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:44.471671 12950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:44.474821 12950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "6c44644c59864452b6ea9ae5afa90778"
format_stamp: "Formatted at 2026-09-26 19:33:44 on dist-test-slave-phbh"
server_key: "186319dd1e87454acd7af54ec88578a4"
server_key_iv: "5410c7f430914d1fd9170af629d9e700"
server_key_version: "encryptionkey@0"
I20260926 19:33:44.475575 12950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "6c44644c59864452b6ea9ae5afa90778"
format_stamp: "Formatted at 2026-09-26 19:33:44 on dist-test-slave-phbh"
server_key: "186319dd1e87454acd7af54ec88578a4"
server_key_iv: "5410c7f430914d1fd9170af629d9e700"
server_key_version: "encryptionkey@0"
I20260926 19:33:44.480574 12950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 19:33:44.484032 12965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:44.485351 12950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:44.485522 12950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "6c44644c59864452b6ea9ae5afa90778"
format_stamp: "Formatted at 2026-09-26 19:33:44 on dist-test-slave-phbh"
server_key: "186319dd1e87454acd7af54ec88578a4"
server_key_iv: "5410c7f430914d1fd9170af629d9e700"
server_key_version: "encryptionkey@0"
I20260926 19:33:44.485728 12950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:44.503196 12950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:44.507035 12950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:44.507329 12950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:44.508244 12950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:44.509600 12950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:44.509702 12950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:44.509853 12950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:44.509925 12950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:44.544937 12950 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:38259
I20260926 19:33:44.544906 13077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:38259 every 8 connection(s)
I20260926 19:33:44.546366 12950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:44.548431 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 12950
I20260926 19:33:44.548625 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:44.549093 7007 external_mini_cluster.cc:1428] Setting key 324933f734ad6f60e750df64e2af528e
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:33:44.568807 12813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.549505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:45337 (local address 127.6.215.254:41623)
0926 19:33:44.549698 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.549704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.551744 (+ 2040us) server_negotiation.cc:407] Connection header received
0926 19:33:44.553872 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.553877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.554032 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.554139 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.555563 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.556462 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.558266 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.558705 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.558858 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.562580 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.562585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.562588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.562619 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.565759 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.566462 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.566467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.566471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.566610 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.567100 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.567104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.567106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.567362 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.567533 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.568223 (+ 690us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.568438 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 19:33:44.569730 13080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.550167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41623 (local address 127.6.215.195:45337)
0926 19:33:44.551598 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:44.551645 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:44.553593 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:44.554345 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:44.554352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:44.554838 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:44.555386 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:44.555402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.556654 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.556662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:44.558098 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:44.558110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.558966 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.558974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:44.559241 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.559906 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:44.559934 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:44.562357 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:44.565905 (+ 3548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:44.565913 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:44.565930 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:44.566330 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:44.566718 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:44.566723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:44.566727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:44.566917 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:44.567499 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:44.567506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:44.567850 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 19:33:44.568125 (+ 275us) client_negotiation.cc:240] Negotiation successful
0926 19:33:44.568642 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":899,"threads_started":1}
I20260926 19:33:44.570981 13078 heartbeater.cc:347] Connected to a master server at 127.6.215.254:41623
I20260926 19:33:44.571280 13078 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:44.572000 13078 heartbeater.cc:511] Master 127.6.215.254:41623 requested a full tablet report, sending...
I20260926 19:33:44.573539 12622 ts_manager.cc:194] Registered new tserver with Master: 6c44644c59864452b6ea9ae5afa90778 (127.6.215.195:38259)
I20260926 19:33:44.574343 12622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:45337
I20260926 19:33:44.577098 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:33:44.583982 12811 heartbeater.cc:503] Master 127.6.215.254:41623 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/test-admin.keytab.
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.605792 7007 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 19:33:44.611328 13085 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:44.608737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.6.215.193:39751)
0926 19:33:44.609456 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.609463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.609489 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:33:44.609547 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.609554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.609761 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.609936 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.610874 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36796: BlockingRecv error: recv got EOF from 127.0.0.1:36796 (error 108)
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":350,"threads_started":1}
W20260926 19:33:44.611546 7007 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5cc' of type FILE
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.613387 7007 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.615100 7007 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:44.630884 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.616809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.6.215.193:39751)
0926 19:33:44.617065 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.617072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.617334 (+ 262us) server_negotiation.cc:407] Connection header received
0926 19:33:44.617557 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.617564 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.617636 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.617745 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.618786 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.619740 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.621142 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.621714 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.621861 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.625598 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.625611 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.625628 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.625663 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.628379 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.629114 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.629121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.629123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.629241 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.629563 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.629566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.629568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.629904 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.630137 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.630418 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.630634 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 19:33:44.636875 13085 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:44.635011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36812 (local address 127.6.215.193:39751)
0926 19:33:44.635217 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.635223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.635435 (+ 212us) server_negotiation.cc:407] Connection header received
0926 19:33:44.635667 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.635671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.635744 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.635847 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.636556 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36812: BlockingRecv error: recv got EOF from 127.0.0.1:36812 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 19:33:44.637149 7007 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5cc' of type FILE
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.638934 7007 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.640300 7007 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:44.654486 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.641684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.6.215.193:39751)
0926 19:33:44.641916 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.641922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.642092 (+ 170us) server_negotiation.cc:407] Connection header received
0926 19:33:44.642218 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.642222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.642277 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.642356 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.643226 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.643951 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.645206 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.645651 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.645752 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.649332 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.649337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.649340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.649389 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.651996 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.652627 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.652631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.652633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.652751 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.653173 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.653179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.653181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.653555 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.653768 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.653966 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.654174 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260926 19:33:44.658828 13085 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:44.657234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.6.215.193:39751)
0926 19:33:44.657414 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.657420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.657542 (+ 122us) server_negotiation.cc:407] Connection header received
0926 19:33:44.657662 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.657665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.657728 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.657801 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.658560 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36834: BlockingRecv error: recv got EOF from 127.0.0.1:36834 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 19:33:44.658867 7007 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790451176367082-7007-0/krb5kdc/krb5cc' of type FILE
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.660383 7007 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:44.661756 7007 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:33:44.676322 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.662989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.6.215.193:39751)
0926 19:33:44.663228 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.663235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.663377 (+ 142us) server_negotiation.cc:407] Connection header received
0926 19:33:44.663493 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.663497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.663549 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.663623 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.664517 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.665317 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.666714 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.667179 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.667337 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.671014 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.671020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.671023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.671054 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.673807 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.674580 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.674584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.674586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.674733 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.675102 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.675105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.675107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.675382 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.675609 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.675796 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.675963 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:33:44.678164 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12683
I20260926 19:33:44.690122 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.678864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36858 (local address 127.6.215.193:39751)
0926 19:33:44.679076 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.679082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.679321 (+ 239us) server_negotiation.cc:407] Connection header received
0926 19:33:44.679419 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.679423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.679477 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.679553 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.680475 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.681276 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.682400 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.682826 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.682954 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.684966 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.684971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.684974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.685002 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.687336 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.687987 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.687991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.687993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.688133 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.688475 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.688480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.688483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.688835 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.689059 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.689526 (+ 467us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.689721 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:33:44.690696 12806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:44.773088 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12683
I20260926 19:33:44.789844 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12817
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:33:44.805860 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.790856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39220 (local address 127.6.215.194:34683)
0926 19:33:44.791458 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.791465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.791490 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:33:44.791554 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.791560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.791767 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.791894 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.792752 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.793820 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.795009 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.795612 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.795718 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.800197 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.800209 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.800224 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.800258 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.803065 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.803734 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.803742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.803744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.803859 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.804242 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.804249 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.804252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.804602 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.804943 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.805278 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.805549 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":322,"threads_started":1}
I20260926 19:33:44.807168 12940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:44.890157 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12817
I20260926 19:33:44.906147 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12950
Sep 26 19:33:44 dist-test-slave-phbh krb5kdc[12577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:33:44.923274 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:44.907130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.6.215.195:38259)
0926 19:33:44.907763 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:44.907770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:44.907797 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:33:44.907861 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:44.907869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:44.908242 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:44.908424 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:44.909471 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.911076 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.912376 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:44.913323 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:44.913454 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:44.917580 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:44.917593 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:44.917609 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:44.917645 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:44.920502 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.921393 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.921400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.921402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.921531 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:44.921909 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:44.921912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:44.921914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:44.922252 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:44.922479 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:44.922799 (+ 320us) server_negotiation.cc:299] Negotiation successful
0926 19:33:44.922994 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":339,"threads_started":1}
I20260926 19:33:44.924561 13073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:45.008280 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12950
I20260926 19:33:45.024341 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 12590
I20260926 19:33:45.025561 12652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:45.127086 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 12590
2026-09-26T19:33:45Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3930 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:45 dist-test-slave-phbh krb5kdc[13105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:45 dist-test-slave-phbh krb5kdc[13105](info): set up 2 sockets
Sep 26 19:33:45 dist-test-slave-phbh krb5kdc[13105](info): commencing operation
krb5kdc: starting...
W20260926 19:33:46.221894 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:46 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:46Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab.
Entry for principal oryx/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab.
I20260926 19:33:46.437676 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:45377
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab
--principal=oryx/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:45377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:46.763530 13118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:46.763820 13118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:46.764014 13118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:46.771302 13118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:46.771411 13118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:46.771516 13118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:46.771618 13118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:46.771698 13118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:46.781566 13118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:45377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab
--principal=oryx/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:45377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:46.782843 13118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:46.784159 13118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:46.795435 13126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:46.795463 13124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:46.795717 13123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:46.796693 13118 server_base.cc:1034] running on GCE node
I20260926 19:33:46.797528 13118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:46.798933 13118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:46.800168 13118 hybrid_clock.cc:648] HybridClock initialized: now 1790451226800116 us; error 45 us; skew 500 ppm
I20260926 19:33:46.800508 13118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:46 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:46.803503 13118 init.cc:377] Logged in from keytab as oryx/127.6.215.254@KRBTEST.COM (short username oryx)
I20260926 19:33:46.805015 13118 webserver.cc:460] Webserver started at http://127.6.215.254:41065/ using document root <none> and password file <none>
I20260926 19:33:46.805693 13118 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:46.805837 13118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:46.806136 13118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:46.809033 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "cd34298261224bda8de2cb4ffa00ce2f"
format_stamp: "Formatted at 2026-09-26 19:33:46 on dist-test-slave-phbh"
server_key: "ceefb4a5a61287f82e0732354601e07e"
server_key_iv: "bc553af869f6dc3feef0724cde93425d"
server_key_version: "encryptionkey@0"
I20260926 19:33:46.809767 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "cd34298261224bda8de2cb4ffa00ce2f"
format_stamp: "Formatted at 2026-09-26 19:33:46 on dist-test-slave-phbh"
server_key: "ceefb4a5a61287f82e0732354601e07e"
server_key_iv: "bc553af869f6dc3feef0724cde93425d"
server_key_version: "encryptionkey@0"
I20260926 19:33:46.814663 13118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260926 19:33:46.818295 13133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:46.819777 13118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 19:33:46.819955 13118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "cd34298261224bda8de2cb4ffa00ce2f"
format_stamp: "Formatted at 2026-09-26 19:33:46 on dist-test-slave-phbh"
server_key: "ceefb4a5a61287f82e0732354601e07e"
server_key_iv: "bc553af869f6dc3feef0724cde93425d"
server_key_version: "encryptionkey@0"
I20260926 19:33:46.820133 13118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:46.835052 13118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:46.839286 13118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:46.839591 13118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:46.858223 13118 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:45377
I20260926 19:33:46.858254 13184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:45377 every 8 connection(s)
I20260926 19:33:46.859628 13118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:46.863401 13185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:46.868567 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 13118
I20260926 19:33:46.868815 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:46.869607 7007 external_mini_cluster.cc:1428] Setting key e4c59e8f8c38add2042d181f6c2bca54
I20260926 19:33:46.875236 13185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: Bootstrap starting.
I20260926 19:33:46.878369 13185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:46.879366 13185 log.cc:824] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:46.882211 13185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: No bootstrap required, opened a new log
Sep 26 19:33:46 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:46.889680 13185 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } }
I20260926 19:33:46.890056 13185 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:46.890187 13185 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd34298261224bda8de2cb4ffa00ce2f, State: Initialized, Role: FOLLOWER
I20260926 19:33:46.890821 13185 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } }
I20260926 19:33:46.891060 13185 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:46.891202 13185 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:46.891398 13185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:46.891963 13188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:46.871888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55100 (local address 127.6.215.254:45377)
0926 19:33:46.872917 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:46.872936 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:46.872983 (+ 47us) server_negotiation.cc:407] Connection header received
0926 19:33:46.874733 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:46.874756 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:46.875176 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:46.875610 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:46.877162 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:46.878510 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:46.879773 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:46.880504 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:46.880696 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:46.884542 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:46.884550 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:46.884566 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:46.884599 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:46.887757 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:46.888442 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:46.888448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:46.888458 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:46.888614 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:46.889020 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:46.889026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:46.889030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:46.889548 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:46.889872 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:46.890309 (+ 437us) server_negotiation.cc:299] Negotiation successful
0926 19:33:46.890762 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":682,"thread_start_us":388,"threads_started":1}
I20260926 19:33:46.894276 13185 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } }
I20260926 19:33:46.894855 13185 leader_election.cc:302] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd34298261224bda8de2cb4ffa00ce2f; no voters:
I20260926 19:33:46.895437 13185 leader_election.cc:288] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:46.895597 13190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:46.896265 13190 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 1 LEADER]: Becoming Leader. State: Replica: cd34298261224bda8de2cb4ffa00ce2f, State: Running, Role: LEADER
I20260926 19:33:46.896817 13190 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } }
I20260926 19:33:46.897569 13185 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:46.899698 13192 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd34298261224bda8de2cb4ffa00ce2f. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } }
I20260926 19:33:46.899646 13191 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } }
I20260926 19:33:46.900091 13191 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:46.900091 13192 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:46.901007 13198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:46.906440 13198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:46.915126 13198 catalog_manager.cc:1347] Generated new cluster ID: c865ba0a47534565a1cbb132a5047d56
I20260926 19:33:46.915248 13198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:46.960503 13198 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:46.961874 13198 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:46.969465 13198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: Generated new TSK 0
I20260926 19:33:46.970361 13198 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab.
Entry for principal oryx/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab.
I20260926 19:33:47.020275 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45377
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab
--principal=oryx/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:47.346922 13211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:47.347215 13211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:47.347379 13211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:47.354873 13211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:47.354990 13211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:47.355223 13211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:47.365065 13211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab
--principal=oryx/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:47.366364 13211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:47.367658 13211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:47.381381 13217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:47.381460 13219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:47.381465 13216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:47.382285 13211 server_base.cc:1034] running on GCE node
I20260926 19:33:47.382933 13211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:47.383890 13211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:47.385095 13211 hybrid_clock.cc:648] HybridClock initialized: now 1790451227385056 us; error 38 us; skew 500 ppm
I20260926 19:33:47.385411 13211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:47 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451227, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:47.388923 13211 init.cc:377] Logged in from keytab as oryx/127.6.215.193@KRBTEST.COM (short username oryx)
I20260926 19:33:47.390412 13211 webserver.cc:460] Webserver started at http://127.6.215.193:38325/ using document root <none> and password file <none>
I20260926 19:33:47.391330 13211 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:47.391443 13211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:47.391688 13211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:47.394718 13211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "f948ce0c2b6149528c17d39f278cfe76"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "8e5850da3073e7d2cd5d192feddb8439"
server_key_iv: "e9c9a856ca1f28b8058ebfcd1aa280b9"
server_key_version: "encryptionkey@0"
I20260926 19:33:47.395407 13211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "f948ce0c2b6149528c17d39f278cfe76"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "8e5850da3073e7d2cd5d192feddb8439"
server_key_iv: "e9c9a856ca1f28b8058ebfcd1aa280b9"
server_key_version: "encryptionkey@0"
I20260926 19:33:47.400641 13211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:33:47.404106 13226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:47.405608 13211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:47.405774 13211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "f948ce0c2b6149528c17d39f278cfe76"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "8e5850da3073e7d2cd5d192feddb8439"
server_key_iv: "e9c9a856ca1f28b8058ebfcd1aa280b9"
server_key_version: "encryptionkey@0"
I20260926 19:33:47.405985 13211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:47.427899 13211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:47.432267 13211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:47.432574 13211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:47.433621 13211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:47.434993 13211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:47.435101 13211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:47.435252 13211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:47.435324 13211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:47.469694 13211 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:34191
I20260926 19:33:47.469712 13338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:34191 every 8 connection(s)
I20260926 19:33:47.470995 13211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:47.471807 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 13211
I20260926 19:33:47.472084 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:47.472858 7007 external_mini_cluster.cc:1428] Setting key a4727af01a59cdf8e7773305c7f1ae13
Sep 26 19:33:47 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451227, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:47.493022 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:47.473750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:42265 (local address 127.6.215.254:45377)
0926 19:33:47.474321 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:47.474325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:47.475550 (+ 1225us) server_negotiation.cc:407] Connection header received
0926 19:33:47.477478 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:47.477485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:47.477577 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:47.477740 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:47.479321 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.480520 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.482178 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.482590 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.482766 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:47.486656 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:47.486662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:47.486665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:47.486696 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:47.489470 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:47.490476 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:47.490481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:47.490483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:47.490596 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:47.491124 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:47.491127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:47.491129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:47.491386 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:47.491492 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:47.492480 (+ 988us) server_negotiation.cc:299] Negotiation successful
0926 19:33:47.492652 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":349,"threads_started":1}
WARNING: no policy specified for oryx/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:33:47.494442 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:47.474475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.193:42265)
0926 19:33:47.475378 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:47.475424 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:47.477203 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:47.478006 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:47.478015 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:47.478540 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:47.479144 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:47.479162 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.480662 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.480666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:47.482017 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:47.482028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.482875 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.482882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:47.483141 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:47.483900 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:47.483928 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:47.486432 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:47.489645 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:47.489656 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:47.489679 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:47.490337 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:47.490710 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:47.490718 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:47.490724 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:47.490998 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:47.491515 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:47.491524 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:47.491999 (+ 475us) client_negotiation.cc:769] Sending connection context
0926 19:33:47.492338 (+ 339us) client_negotiation.cc:240] Negotiation successful
0926 19:33:47.492926 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":383,"threads_started":1}
I20260926 19:33:47.496028 13339 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45377
I20260926 19:33:47.496335 13339 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.194@KRBTEST.COM" created.
I20260926 19:33:47.497138 13339 heartbeater.cc:511] Master 127.6.215.254:45377 requested a full tablet report, sending...
I20260926 19:33:47.499485 13150 ts_manager.cc:194] Registered new tserver with Master: f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191)
I20260926 19:33:47.500962 13150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:42265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab.
Entry for principal oryx/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab.
I20260926 19:33:47.524178 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45377
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab
--principal=oryx/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:47.842866 13345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:47.843149 13345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:47.843242 13345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:47.850423 13345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:47.850545 13345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:47.850656 13345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:47.860230 13345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab
--principal=oryx/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:47.861521 13345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:47.862756 13345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:47.876020 13350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:47.876012 13351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:47.876204 13353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:47.876875 13345 server_base.cc:1034] running on GCE node
I20260926 19:33:47.877489 13345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:47.878499 13345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:47.879724 13345 hybrid_clock.cc:648] HybridClock initialized: now 1790451227879679 us; error 39 us; skew 500 ppm
I20260926 19:33:47.880080 13345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:47 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451227, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:47.883133 13345 init.cc:377] Logged in from keytab as oryx/127.6.215.194@KRBTEST.COM (short username oryx)
I20260926 19:33:47.884579 13345 webserver.cc:460] Webserver started at http://127.6.215.194:37433/ using document root <none> and password file <none>
I20260926 19:33:47.885317 13345 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:47.885459 13345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:47.885753 13345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:47.888568 13345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "07f3f61459da425bb46983f15833d8a4"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "d090557af7819301fdaabfa838c3df25"
server_key_iv: "1c035d0ad6cb05920a682b04b1927c03"
server_key_version: "encryptionkey@0"
I20260926 19:33:47.889372 13345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "07f3f61459da425bb46983f15833d8a4"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "d090557af7819301fdaabfa838c3df25"
server_key_iv: "1c035d0ad6cb05920a682b04b1927c03"
server_key_version: "encryptionkey@0"
I20260926 19:33:47.894438 13345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:33:47.898000 13360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:47.899283 13345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:47.899448 13345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "07f3f61459da425bb46983f15833d8a4"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "d090557af7819301fdaabfa838c3df25"
server_key_iv: "1c035d0ad6cb05920a682b04b1927c03"
server_key_version: "encryptionkey@0"
I20260926 19:33:47.899641 13345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:47.913390 13345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:47.917296 13345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:47.917589 13345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:47.918545 13345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:47.919902 13345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:47.920010 13345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:47.920154 13345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:47.920229 13345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:47.954702 13345 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:45579
I20260926 19:33:47.954788 13472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:45579 every 8 connection(s)
I20260926 19:33:47.955991 13345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:47.964453 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 13345
I20260926 19:33:47.964648 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:47.965117 7007 external_mini_cluster.cc:1428] Setting key faba7f50ddabb92bd780958212e9f50f
Sep 26 19:33:47 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451227, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:47.977638 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:47.958607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:40429 (local address 127.6.215.254:45377)
0926 19:33:47.958790 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:47.958794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:47.959859 (+ 1065us) server_negotiation.cc:407] Connection header received
0926 19:33:47.961752 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:47.961758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:47.961851 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:47.961965 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:47.963453 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.964188 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.966076 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.966511 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.966649 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:47.970858 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:47.970867 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:47.970875 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:47.970921 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:47.974255 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:47.975242 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:47.975249 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:47.975252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:47.975432 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:47.975866 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:47.975871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:47.975875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:47.976149 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:47.976299 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:47.977073 (+ 774us) server_negotiation.cc:299] Negotiation successful
0926 19:33:47.977293 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:33:47.978680 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:47.959006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:40429)
0926 19:33:47.959700 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:47.959737 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:47.961469 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:47.962204 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:47.962212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:47.962721 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:47.963286 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:47.963303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.964338 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.964350 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:47.965927 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:47.965939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:47.966736 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:47.966742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:47.967021 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:47.967781 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:47.967815 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:47.970618 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:47.974420 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:47.974432 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:47.974452 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:47.975109 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:47.975547 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:47.975552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:47.975555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:47.975744 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:47.976274 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:47.976282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:47.976636 (+ 354us) client_negotiation.cc:769] Sending connection context
0926 19:33:47.976970 (+ 334us) client_negotiation.cc:240] Negotiation successful
0926 19:33:47.977532 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":262,"threads_started":1}
I20260926 19:33:47.980145 13473 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45377
I20260926 19:33:47.980463 13473 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:47.981222 13473 heartbeater.cc:511] Master 127.6.215.254:45377 requested a full tablet report, sending...
I20260926 19:33:47.982621 13150 ts_manager.cc:194] Registered new tserver with Master: 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579)
I20260926 19:33:47.983409 13150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:40429
WARNING: no policy specified for oryx/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab.
Entry for principal oryx/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab.
I20260926 19:33:48.016595 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45377
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab
--principal=oryx/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:48.343940 13478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:48.344298 13478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:48.344477 13478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:48.351943 13478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:48.352120 13478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:48.352313 13478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:48.362344 13478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab
--principal=oryx/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:48.363739 13478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:48.365185 13478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:48.378909 13483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:48.378980 13486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:48.378942 13484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:48.379740 13478 server_base.cc:1034] running on GCE node
I20260926 19:33:48.380411 13478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:48.381451 13478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:48.382663 13478 hybrid_clock.cc:648] HybridClock initialized: now 1790451228382609 us; error 44 us; skew 500 ppm
I20260926 19:33:48.382954 13478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:48 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451228, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:48.386246 13478 init.cc:377] Logged in from keytab as oryx/127.6.215.195@KRBTEST.COM (short username oryx)
I20260926 19:33:48.387846 13478 webserver.cc:460] Webserver started at http://127.6.215.195:41269/ using document root <none> and password file <none>
I20260926 19:33:48.388594 13478 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:48.388746 13478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:48.389096 13478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:48.392046 13478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "361e1a07f3014ab2b12ce6ff07fc3164"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "76a8b1629b9c34fe7eaaa1573b7e61d5"
server_key_iv: "95831cf6691b4b9c07cd65e06db72f02"
server_key_version: "encryptionkey@0"
I20260926 19:33:48.392899 13478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "361e1a07f3014ab2b12ce6ff07fc3164"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "76a8b1629b9c34fe7eaaa1573b7e61d5"
server_key_iv: "95831cf6691b4b9c07cd65e06db72f02"
server_key_version: "encryptionkey@0"
I20260926 19:33:48.398485 13478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:33:48.402192 13493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:48.403640 13478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 19:33:48.403818 13478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "361e1a07f3014ab2b12ce6ff07fc3164"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "76a8b1629b9c34fe7eaaa1573b7e61d5"
server_key_iv: "95831cf6691b4b9c07cd65e06db72f02"
server_key_version: "encryptionkey@0"
I20260926 19:33:48.404079 13478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:48.444517 13478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:48.449409 13478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:48.449750 13478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:48.450775 13478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:48.452214 13478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:48.452322 13478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:48.452476 13478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:48.452549 13478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:48.487602 13478 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:36411
I20260926 19:33:48.487653 13605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:36411 every 8 connection(s)
I20260926 19:33:48.488974 13478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:48.497440 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 13478
I20260926 19:33:48.497648 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:48.498242 7007 external_mini_cluster.cc:1428] Setting key 5c829b48b1b61ed454808b7d11544bff
Sep 26 19:33:48 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451228, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:48.504343 13339 heartbeater.cc:503] Master 127.6.215.254:45377 was elected leader, sending a full tablet report...
I20260926 19:33:48.510196 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:48.491591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:59925 (local address 127.6.215.254:45377)
0926 19:33:48.492063 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:48.492070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:48.493110 (+ 1040us) server_negotiation.cc:407] Connection header received
0926 19:33:48.494838 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:48.494843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:48.494932 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:48.495039 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:48.496458 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:48.497438 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:48.499568 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:48.500041 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:48.500165 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:48.503790 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:48.503794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:48.503797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:48.503828 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:48.507145 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:48.507896 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:48.507900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:48.507902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:48.508018 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:48.508501 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:48.508505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:48.508507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:48.508804 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:48.508943 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:48.509705 (+ 762us) server_negotiation.cc:299] Negotiation successful
0926 19:33:48.509876 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":238,"threads_started":1}
I20260926 19:33:48.511269 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:48.492181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.195:59925)
0926 19:33:48.492941 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:48.492985 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:48.494565 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:48.495241 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:48.495250 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:48.495752 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:48.496296 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:48.496312 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:48.497603 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:48.497608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:48.499433 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:48.499446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:48.500254 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:48.500259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:48.500554 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:48.501421 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:48.501459 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:48.503593 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:48.507288 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:48.507302 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:48.507321 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:48.507784 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:48.508157 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:48.508162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:48.508169 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:48.508382 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:48.508929 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:48.508935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:48.509306 (+ 371us) client_negotiation.cc:769] Sending connection context
0926 19:33:48.509587 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 19:33:48.510007 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":287,"threads_started":1}
I20260926 19:33:48.512868 13606 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45377
I20260926 19:33:48.513233 13606 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:48.514012 13606 heartbeater.cc:511] Master 127.6.215.254:45377 requested a full tablet report, sending...
I20260926 19:33:48.515379 13150 ts_manager.cc:194] Registered new tserver with Master: 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:48.516177 13150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:59925
I20260926 19:33:48.526347 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.215.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.253.keytab.
Entry for principal oryx/127.6.215.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.253.keytab.
I20260926 19:33:48.577256 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:33967
--webserver_interface=127.6.215.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.253.keytab
--principal=oryx/127.6.215.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:45377,127.6.215.253:33967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:48.915618 13612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:48.915905 13612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:48.915997 13612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:48.923481 13612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:48.923601 13612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:48.923648 13612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:48.923738 13612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:48.923808 13612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:48.933718 13612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:45377,127.6.215.253:33967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.253.keytab
--principal=oryx/127.6.215.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.215.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
W20260926 19:33:48.934134 13612 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.215.254:45377,127.6.215.253:33967'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 19:33:48.935201 13612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:48.936517 13612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:48.948447 13618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:48.948544 13617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:48.948518 13620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:48.949674 13612 server_base.cc:1034] running on GCE node
I20260926 19:33:48.950448 13612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:48.951872 13612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:48.953092 13612 hybrid_clock.cc:648] HybridClock initialized: now 1790451228953034 us; error 38 us; skew 500 ppm
I20260926 19:33:48.953389 13612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:48 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451228, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:48.956597 13612 init.cc:377] Logged in from keytab as oryx/127.6.215.253@KRBTEST.COM (short username oryx)
I20260926 19:33:48.958364 13612 webserver.cc:460] Webserver started at http://127.6.215.253:40683/ using document root <none> and password file <none>
I20260926 19:33:48.959048 13612 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:48.959132 13612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:48.959431 13612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:48.962393 13612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/instance:
uuid: "d44b3986271c40009d8c47e02ff35329"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "61a06b4231d5e79923759b3a14a26e01"
server_key_iv: "adcbc4cb43b4f78ab1c4d6e5e54559f1"
server_key_version: "encryptionkey@0"
I20260926 19:33:48.963165 13612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal/instance:
uuid: "d44b3986271c40009d8c47e02ff35329"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "61a06b4231d5e79923759b3a14a26e01"
server_key_iv: "adcbc4cb43b4f78ab1c4d6e5e54559f1"
server_key_version: "encryptionkey@0"
I20260926 19:33:48.968225 13612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 19:33:48.971529 13627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:48.972757 13612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 19:33:48.972997 13612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
uuid: "d44b3986271c40009d8c47e02ff35329"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "61a06b4231d5e79923759b3a14a26e01"
server_key_iv: "adcbc4cb43b4f78ab1c4d6e5e54559f1"
server_key_version: "encryptionkey@0"
I20260926 19:33:48.973220 13612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:48.986824 13473 heartbeater.cc:503] Master 127.6.215.254:45377 was elected leader, sending a full tablet report...
I20260926 19:33:48.991689 13612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:48.996057 13612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:48.996316 13612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:49.015677 13612 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.253:33967
I20260926 19:33:49.015748 13678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.253:33967 every 8 connection(s)
I20260926 19:33:49.017537 13612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
I20260926 19:33:49.022051 13679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:49.025698 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 13612
I20260926 19:33:49.030961 13679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.253@KRBTEST.COM
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451228, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.253@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:49.050482 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.033657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.6.215.254:45377)
0926 19:33:49.034956 (+ 1299us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.034961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.034980 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:33:49.035037 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.035042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.035119 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.035230 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.036695 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.037820 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.040173 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.040553 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.040652 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.044500 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.044506 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.044509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.044540 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.047226 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.048161 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.048169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.048173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.048313 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.048697 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.048701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.048704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.049143 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.049276 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.049921 (+ 645us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.050151 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1177,"thread_start_us":1066,"threads_started":1}
I20260926 19:33:49.051393 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.028277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48956 (local address 127.6.215.253:33967)
0926 19:33:49.029110 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.029122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.029161 (+ 39us) server_negotiation.cc:407] Connection header received
0926 19:33:49.030758 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.030779 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.031172 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.031574 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.033185 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.035074 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.037049 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.038049 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.038176 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.043476 (+ 5300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.043488 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.043507 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.043555 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.046767 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.047538 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.047544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.047547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.047711 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.048187 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.048192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.048195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.048996 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.049300 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.049767 (+ 467us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.050212 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":633,"spinlock_wait_cycles":17792,"thread_start_us":390,"threads_started":1}
I20260926 19:33:49.051508 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.033599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55112)
0926 19:33:49.034294 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.034334 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.034493 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.035366 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.035374 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.036220 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.036538 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.036549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.037999 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.038003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.040010 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.040029 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.040831 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.040843 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.041072 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.041731 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.041768 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.044293 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.047398 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.047411 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.047424 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.048013 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.048429 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.048432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.048435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.048582 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.049277 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.049286 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.049555 (+ 269us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.049828 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.050372 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":102,"thread_start_us":291,"threads_started":1}
I20260926 19:33:49.053416 13679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451228, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.253@KRBTEST.COM for oryx/127.6.215.253@KRBTEST.COM
I20260926 19:33:49.065747 13683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.054080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:48960)
0926 19:33:49.054254 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.054275 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.054400 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.054776 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.054780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.055134 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.055278 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.055286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.056212 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.056216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.057383 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.057396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.058037 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.058042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.058227 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.058776 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.058793 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.060728 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.063374 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.063382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.063386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.063745 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.064085 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.064091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.064095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.064218 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.064706 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.064710 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.064903 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.065049 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.065362 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":1}
I20260926 19:33:49.065794 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.054310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.6.215.253:33967)
0926 19:33:49.054470 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.054475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.054494 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:33:49.054540 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.054544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.054611 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.054708 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.055396 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.056080 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.057505 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.057905 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.058005 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.060947 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.060952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.060955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.060981 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.063241 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.063856 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.063864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.063868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.063982 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.064324 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.064329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.064332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.064599 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.064742 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.065117 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.065495 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":49}
I20260926 19:33:49.070585 13679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Bootstrap starting.
I20260926 19:33:49.073874 13679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:49.074905 13679 log.cc:824] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:49.077700 13679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: No bootstrap required, opened a new log
I20260926 19:33:49.084661 13679 raft_consensus.cc:348] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } }
I20260926 19:33:49.085055 13679 raft_consensus.cc:374] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:49.085178 13679 raft_consensus.cc:729] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d44b3986271c40009d8c47e02ff35329, State: Initialized, Role: FOLLOWER
I20260926 19:33:49.085749 13679 consensus_queue.cc:260] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } }
I20260926 19:33:49.086622 13686 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } } }
I20260926 19:33:49.086989 13686 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:33:49.087602 13679 sys_catalog.cc:564] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 19:33:49.096669 13697 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:33:49.096844 13697 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:33:49.098552 13612 master_runner.cc:428] Detected that this master d44b3986271c40009d8c47e02ff35329 is joining an existing cluster
I20260926 19:33:49.098663 13612 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.215.253:33967
I20260926 19:33:49.118772 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.102436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.6.215.254:45377)
0926 19:33:49.102636 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.102644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.103326 (+ 682us) server_negotiation.cc:407] Connection header received
0926 19:33:49.103581 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.103587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.103680 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.103753 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.105107 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.106196 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.108429 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.109128 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.109231 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.111752 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.111759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.111765 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.111828 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.115478 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.116342 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.116348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.116352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.116506 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.117021 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.117027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.117030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.117381 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.117508 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.118160 (+ 652us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.118405 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:33:49.119961 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.102504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55120)
0926 19:33:49.103223 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.103239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.103400 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.103953 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.103957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.104481 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.104893 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.104911 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.106507 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.106513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.108255 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.108269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.109520 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.109526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.109697 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.110392 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.110415 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.111539 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.115639 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.115649 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.115654 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.116196 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.116666 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.116671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.116674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.116880 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.117611 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.117618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.117831 (+ 213us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.119094 (+ 1263us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.119451 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":1,"thread_start_us":384,"threads_started":1}
I20260926 19:33:49.122575 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.103587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.6.215.253:33967)
0926 19:33:49.103757 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.103762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.105295 (+ 1533us) server_negotiation.cc:407] Connection header received
0926 19:33:49.105531 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.105535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.105597 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.105654 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.107125 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.108270 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.110512 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.111365 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.111482 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.114462 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.114470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.114476 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.114511 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.118425 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.119558 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.119565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.119571 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.119696 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.120266 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.120274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.120280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.121313 (+ 1033us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.121460 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.121979 (+ 519us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.122203 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260926 19:33:49.123626 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.103491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:48974)
0926 19:33:49.105221 (+ 1730us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.105238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.105405 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.105800 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.105804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.106574 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.106973 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.106986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.108596 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.108604 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.110356 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.110372 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.111886 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.111895 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.112194 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.113074 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.113100 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.114253 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.118574 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.118580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.118587 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.119426 (+ 839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.119819 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.119826 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.119832 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.120007 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.121526 (+ 1519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.121530 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.121685 (+ 155us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.122839 (+ 1154us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.123157 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1605,"thread_start_us":1970,"threads_started":1}
WARNING: no policy specified for oryx/127.6.215.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.252@KRBTEST.COM" created.
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.254@KRBTEST.COM for oryx/127.6.215.253@KRBTEST.COM
I20260926 19:33:49.143033 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.127222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:48982)
0926 19:33:49.127868 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.127918 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.128137 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.128715 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.128724 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.129614 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.129919 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.129933 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.131009 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.131015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.132635 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.132651 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.133587 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.133593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.133792 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.134452 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.134470 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.136533 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.139753 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.139767 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.139775 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.140347 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.140808 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.140813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.140818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.141011 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.141643 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.141654 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.141867 (+ 213us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.142075 (+ 208us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.142477 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":460,"threads_started":1}
I20260926 19:33:49.144251 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.127500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.6.215.253:33967)
0926 19:33:49.127681 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.127688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.128002 (+ 314us) server_negotiation.cc:407] Connection header received
0926 19:33:49.128419 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.128426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.128526 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.128647 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.130101 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.130802 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.132876 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.133419 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.133567 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.136730 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.136735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.136738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.136823 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.139610 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.140485 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.140491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.140495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.140655 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.141145 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.141151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.141154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.141490 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.141632 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.143558 (+ 1926us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.143818 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:33:49.146221 13150 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.215.253:33967: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d44b3986271c40009d8c47e02ff35329"
member_type: NON_VOTER
last_known_addr {
host: "127.6.215.253"
port: 33967
}
attrs {
promote: true
}
}
dest_uuid: "cd34298261224bda8de2cb4ffa00ce2f"
cas_config_opid_index: -1
I20260926 19:33:49.147363 13150 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } }
I20260926 19:33:49.159037 13710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d44b3986271c40009d8c47e02ff35329 (127.6.215.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } }
I20260926 19:33:49.159755 13654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:49.163604 13710 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.215.253:33967
I20260926 19:33:49.164023 13654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Refusing update from remote peer cd34298261224bda8de2cb4ffa00ce2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 19:33:49.164100 13709 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } }
I20260926 19:33:49.164438 13709 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:49.164656 13612 master.cc:537] Master@127.6.215.253:33967 shutting down...
I20260926 19:33:49.165549 13710 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.252.keytab.
Entry for principal oryx/127.6.215.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.252.keytab.
I20260926 19:33:49.169248 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:34391
--webserver_interface=127.6.215.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.252.keytab
--principal=oryx/127.6.215.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:33:49.177681 13612 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:49.178169 13612 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:49.178381 13612 tablet_replica.cc:330] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: stopping tablet replica
W20260926 19:33:49.179075 13135 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260926 19:33:49.185542 13612 master.cc:552] Master@127.6.215.253:33967 shutdown complete.
I20260926 19:33:49.185628 13612 master_runner.cc:305] Clearing existing system tablet
I20260926 19:33:49.186041 13612 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:49.186133 13612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:49.190258 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.189564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:48990)
0926 19:33:49.189775 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.190034 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 19:33:49.190779 13135 consensus_peers.cc:487] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f -> Peer d44b3986271c40009d8c47e02ff35329 (127.6.215.253:33967): Couldn't send request to peer d44b3986271c40009d8c47e02ff35329. Status: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33967: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 19:33:49.193022 13612 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.006s
I20260926 19:33:49.194705 13716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:49.195145 13612 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:49.195468 13612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260926 19:33:49.195578 13612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
uuid: "d44b3986271c40009d8c47e02ff35329"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "61a06b4231d5e79923759b3a14a26e01"
server_key_iv: "adcbc4cb43b4f78ab1c4d6e5e54559f1"
server_key_version: "encryptionkey@0"
I20260926 19:33:49.201454 13612 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:49.208653 13612 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 19:33:49.208899 13612 log.cc:1196] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 19:33:49.209386 13612 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Deleting consensus metadata
I20260926 19:33:49.209679 13612 master_runner.cc:315] Copying system tablet from 127.6.215.254:45377
I20260926 19:33:49.214179 13612 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: tablet copy: Beginning tablet copy session from remote peer at address 127.6.215.254:45377
I20260926 19:33:49.232268 13684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.215060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.6.215.254:45377)
0926 19:33:49.215240 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.215246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.217340 (+ 2094us) server_negotiation.cc:407] Connection header received
0926 19:33:49.217616 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.217621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.217711 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.217781 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.219012 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.220133 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.222051 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.222731 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.222829 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.225680 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.225687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.225693 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.225731 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.229271 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.230063 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.230069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.230073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.230207 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.230637 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.230643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.230646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.230979 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.231106 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.231664 (+ 558us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.231886 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:33:49.233448 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.215021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55134)
0926 19:33:49.217212 (+ 2191us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.217237 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.217465 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.218015 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.218020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.218568 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.218838 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.218855 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.220317 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.220322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.221880 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.221894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.223092 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.223098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.223339 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.224183 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.224206 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.225454 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.229443 (+ 3989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.229449 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.229454 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.229927 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.230361 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.230366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.230369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.230516 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.231184 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.231189 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.231377 (+ 188us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.232655 (+ 1278us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.232984 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2027,"thread_start_us":1931,"threads_started":1}
I20260926 19:33:49.234616 13170 tablet_copy_service.cc:140] P cd34298261224bda8de2cb4ffa00ce2f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d44b3986271c40009d8c47e02ff35329 ({username='oryx', principal='oryx/127.6.215.253@KRBTEST.COM'} at 127.0.0.1:55134)
I20260926 19:33:49.234992 13170 tablet_copy_service.cc:161] P cd34298261224bda8de2cb4ffa00ce2f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d44b3986271c40009d8c47e02ff35329 at {username='oryx', principal='oryx/127.6.215.253@KRBTEST.COM'} at 127.0.0.1:55134: session id = d44b3986271c40009d8c47e02ff35329-00000000000000000000000000000000
I20260926 19:33:49.240403 13170 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 19:33:49.242776 13612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 19:33:49.249846 13612 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: tablet copy: Starting download of 0 data blocks...
I20260926 19:33:49.250170 13612 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: tablet copy: Starting download of 1 WAL segments...
I20260926 19:33:49.253129 13612 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 19:33:49.257702 13170 tablet_copy_service.cc:342] P cd34298261224bda8de2cb4ffa00ce2f: Request end of tablet copy session d44b3986271c40009d8c47e02ff35329-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.215.253@KRBTEST.COM'} at 127.0.0.1:55134
I20260926 19:33:49.257969 13170 tablet_copy_service.cc:433] P cd34298261224bda8de2cb4ffa00ce2f: ending tablet copy session d44b3986271c40009d8c47e02ff35329-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d44b3986271c40009d8c47e02ff35329
I20260926 19:33:49.268750 13612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:49.271359 13723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:49.281158 13724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:49.281710 13726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:49.285533 13612 server_base.cc:1034] running on GCE node
I20260926 19:33:49.285881 13612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:49.289074 13612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:49.290297 13612 hybrid_clock.cc:648] HybridClock initialized: now 1790451229290231 us; error 48 us; skew 500 ppm
I20260926 19:33:49.290536 13612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451229, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:49.292922 13612 init.cc:377] Logged in from keytab as oryx/127.6.215.253@KRBTEST.COM (short username oryx)
I20260926 19:33:49.294341 13612 webserver.cc:460] Webserver started at http://127.6.215.253:40443/ using document root <none> and password file <none>
I20260926 19:33:49.294759 13612 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:49.294859 13612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:49.298159 13612 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260926 19:33:49.300376 13733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:49.300902 13612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 19:33:49.301093 13612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
uuid: "d44b3986271c40009d8c47e02ff35329"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "61a06b4231d5e79923759b3a14a26e01"
server_key_iv: "adcbc4cb43b4f78ab1c4d6e5e54559f1"
server_key_version: "encryptionkey@0"
I20260926 19:33:49.301302 13612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:49.334751 13612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:49.339641 13612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:49.359761 13612 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.253:33967
I20260926 19:33:49.359834 13784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.253:33967 every 8 connection(s)
I20260926 19:33:49.360951 13612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
I20260926 19:33:49.366706 13785 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:33:49.370950 13785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Bootstrap starting.
I20260926 19:33:49.394541 13785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:33:49.395133 13785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: Bootstrap complete.
I20260926 19:33:49.396163 13785 raft_consensus.cc:348] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } }
I20260926 19:33:49.396421 13785 raft_consensus.cc:729] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d44b3986271c40009d8c47e02ff35329, State: Initialized, Role: LEARNER
I20260926 19:33:49.396643 13785 consensus_queue.cc:260] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } }
I20260926 19:33:49.398128 13785 sys_catalog.cc:564] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:49.397955 13788 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } }
I20260926 19:33:49.398517 13788 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: LEARNER
I20260926 19:33:49.409530 13799 catalog_manager.cc:1259] Loaded cluster ID: c865ba0a47534565a1cbb132a5047d56
I20260926 19:33:49.411747 13799 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: loading cluster ID for follower catalog manager: success
I20260926 19:33:49.415977 13799 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: acquiring CA information for follower catalog manager: success
I20260926 19:33:49.419018 13799 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:33:49.518608 13606 heartbeater.cc:503] Master 127.6.215.254:45377 was elected leader, sending a full tablet report...
I20260926 19:33:49.590144 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.583357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:49004)
0926 19:33:49.583630 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.583651 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.583779 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.585030 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.585037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.585068 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:49.585273 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.585282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.586634 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.586640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.587831 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.587843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.589240 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.589246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.589497 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.589572 (+ 75us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.589682 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.589821 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260926 19:33:49.590485 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.583612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.6.215.253:33967)
0926 19:33:49.584646 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.584650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.584667 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:33:49.584721 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.584725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.584838 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.584995 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:49.585419 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.586488 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.588018 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.589071 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.589216 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.589476 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.589705 (+ 229us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.589830 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":443,"threads_started":1}
I20260926 19:33:49.591905 13760 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260926 19:33:49.592785 13788 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd34298261224bda8de2cb4ffa00ce2f. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } }
I20260926 19:33:49.593044 13788 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: LEARNER
W20260926 19:33:49.652081 13713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:49.652385 13713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:49.652556 13713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:49.660000 13713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:49.660105 13713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:49.660207 13713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:49.660305 13713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:49.660394 13713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:49.670264 13713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.252.keytab
--principal=oryx/127.6.215.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:34391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.215.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:49.671579 13713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:49.672895 13713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:49.684554 13805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:49.684600 13806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:49.684641 13808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:49.685442 13713 server_base.cc:1034] running on GCE node
I20260926 19:33:49.686159 13713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:49.687544 13713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:49.688740 13713 hybrid_clock.cc:648] HybridClock initialized: now 1790451229688699 us; error 39 us; skew 500 ppm
I20260926 19:33:49.689106 13713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451229, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:49.692581 13713 init.cc:377] Logged in from keytab as oryx/127.6.215.252@KRBTEST.COM (short username oryx)
I20260926 19:33:49.694060 13713 webserver.cc:460] Webserver started at http://127.6.215.252:39145/ using document root <none> and password file <none>
I20260926 19:33:49.694741 13713 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:49.694881 13713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:49.695179 13713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:49.698230 13713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/instance:
uuid: "3f9ef2395678483bb14fe6b9aa5ec209"
format_stamp: "Formatted at 2026-09-26 19:33:49 on dist-test-slave-phbh"
server_key: "d5240700546c6039e94e77dc4d357908"
server_key_iv: "1795778c80e9de4cd646d76167bf9b87"
server_key_version: "encryptionkey@0"
I20260926 19:33:49.699059 13713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal/instance:
uuid: "3f9ef2395678483bb14fe6b9aa5ec209"
format_stamp: "Formatted at 2026-09-26 19:33:49 on dist-test-slave-phbh"
server_key: "d5240700546c6039e94e77dc4d357908"
server_key_iv: "1795778c80e9de4cd646d76167bf9b87"
server_key_version: "encryptionkey@0"
I20260926 19:33:49.705084 13713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 19:33:49.709197 13815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:49.710736 13713 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 19:33:49.710914 13713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
uuid: "3f9ef2395678483bb14fe6b9aa5ec209"
format_stamp: "Formatted at 2026-09-26 19:33:49 on dist-test-slave-phbh"
server_key: "d5240700546c6039e94e77dc4d357908"
server_key_iv: "1795778c80e9de4cd646d76167bf9b87"
server_key_version: "encryptionkey@0"
I20260926 19:33:49.711194 13713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:49.737645 13713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:49.742400 13713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:49.742684 13713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:49.762109 13713 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.252:34391
I20260926 19:33:49.762131 13866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.252:34391 every 8 connection(s)
I20260926 19:33:49.763495 13713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
I20260926 19:33:49.767493 13867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:49.771829 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 13713
I20260926 19:33:49.774744 13867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451229, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.252@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.252@KRBTEST.COM
I20260926 19:33:49.795516 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.779312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.6.215.254:45377)
0926 19:33:49.780614 (+ 1302us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.780620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.780639 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:33:49.780694 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.780700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.780807 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.780918 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.782085 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.782773 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.784402 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.785037 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.785132 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.788794 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.788800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.788805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.788850 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.792600 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.793399 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.793404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.793408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.793549 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.793935 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.793941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.793944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.794280 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.794449 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.795066 (+ 617us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.795222 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":645,"threads_started":1}
I20260926 19:33:49.796449 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.779086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55144)
0926 19:33:49.779963 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.780028 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.780186 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.781009 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.781014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.781601 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.781933 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.781945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.782911 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.782918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.784221 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.784236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.785251 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.785258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.785471 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.786071 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.786104 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.788586 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.792738 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.792746 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.792756 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.793277 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.793658 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.793661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.793663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.793820 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.794411 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.794416 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.794723 (+ 307us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.794955 (+ 232us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.795290 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":478,"threads_started":1}
I20260926 19:33:49.798084 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.773974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.6.215.252:34391)
0926 19:33:49.774725 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.774739 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.774777 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:33:49.776578 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.776600 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.777049 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.777359 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.779753 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.781585 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.782994 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.783686 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.783814 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.789678 (+ 5864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.789692 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.789708 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.789742 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.793627 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.795784 (+ 2157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.795790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.795792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.795918 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.796296 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.796303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.796306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.796661 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.796975 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.797344 (+ 369us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.797594 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":345,"threads_started":1}
I20260926 19:33:49.798234 13867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451229, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.252@KRBTEST.COM for oryx/127.6.215.253@KRBTEST.COM
I20260926 19:33:49.811478 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.799096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.6.215.253:33967)
0926 19:33:49.799291 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.799297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.799320 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:33:49.799372 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.799376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.799432 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.799511 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.800326 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.801359 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.802781 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.803190 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.803291 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.806446 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.806451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.806454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.806485 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.809427 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.809981 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.809984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.809987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.810095 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.810390 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.810392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.810394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.810637 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.810740 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.811072 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.811244 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 19:33:49.811482 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.798912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:49020)
0926 19:33:49.799073 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.799092 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.799184 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.799573 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.799577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.799972 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.800143 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.800153 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.801522 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.801526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.802626 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.802636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.803324 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.803329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.803730 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.804301 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.804316 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.806276 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.809539 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.809542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.809545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.809884 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.810183 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.810186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.810188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.810303 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.810727 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.810730 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.810832 (+ 102us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.810933 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.811208 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 19:33:49.812493 13867 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451229, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.252@KRBTEST.COM for oryx/127.6.215.252@KRBTEST.COM
I20260926 19:33:49.824400 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.813361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.0.0.1:43382)
0926 19:33:49.813537 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.813551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.813622 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.813884 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.813887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.814180 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.814310 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.814318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.815337 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.815340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.816266 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.816276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.816955 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.816959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.817105 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.817649 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.817665 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.819508 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.822150 (+ 2642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.822154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.822156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.822473 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.822846 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.822849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.822851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.822983 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.823602 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.823622 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.823752 (+ 130us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.823901 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.824082 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 19:33:49.824438 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.813363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.6.215.252:34391)
0926 19:33:49.813579 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.813583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.813597 (+ 14us) server_negotiation.cc:407] Connection header received
0926 19:33:49.813713 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.813716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.813767 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.813846 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.814455 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.815222 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.816381 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.816822 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.816938 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.819662 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.819666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.819669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.819696 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.822038 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.822597 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.822600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.822603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.822714 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.823206 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.823211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.823218 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.823500 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.823604 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.824001 (+ 397us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.824190 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":145}
I20260926 19:33:49.828950 13867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Bootstrap starting.
I20260926 19:33:49.832291 13867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:49.833304 13867 log.cc:824] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:49.835700 13867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: No bootstrap required, opened a new log
I20260926 19:33:49.842504 13867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } }
I20260926 19:33:49.842787 13867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:49.842862 13867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9ef2395678483bb14fe6b9aa5ec209, State: Initialized, Role: FOLLOWER
I20260926 19:33:49.843323 13867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } }
I20260926 19:33:49.844133 13874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } } }
I20260926 19:33:49.844491 13874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:33:49.845144 13867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:49.852726 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 13211
W20260926 19:33:49.854521 13885 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:33:49.854820 13885 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:33:49.859822 13713 master_runner.cc:428] Detected that this master 3f9ef2395678483bb14fe6b9aa5ec209 is joining an existing cluster
I20260926 19:33:49.859949 13713 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.215.252:34391
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.193@KRBTEST.COM
I20260926 19:33:49.876529 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.854349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.6.215.193:34191)
0926 19:33:49.855802 (+ 1453us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.855811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.855838 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:33:49.855903 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.855913 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.856184 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.856345 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.857668 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.859200 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.861147 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.862022 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.867751 (+ 5729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.868045 (+ 294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.868058 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.868074 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.868107 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.871955 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.872853 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.872862 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.872866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.873027 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.873474 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.873478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.873832 (+ 354us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.874268 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.875675 (+ 1407us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.875836 (+ 161us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.876111 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1275,"thread_start_us":1128,"threads_started":1}
I20260926 19:33:49.878094 13334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:49.926615 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.896195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.6.215.252:34391)
0926 19:33:49.896388 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.896393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.908234 (+ 11841us) server_negotiation.cc:407] Connection header received
0926 19:33:49.909039 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.909045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.909116 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.909179 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.911880 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.912984 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.915547 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.916311 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.916431 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.918816 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.918822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.918826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.918868 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.922989 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.924281 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.924286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.924290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.924412 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.924904 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.924910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.924914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.925281 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.925403 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.926005 (+ 602us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.926207 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":53}
I20260926 19:33:49.927490 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.895514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.6.215.254:45377)
0926 19:33:49.896657 (+ 1143us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.896663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.896689 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:33:49.896736 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.896740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.896862 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.896960 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.906111 (+ 9151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.907587 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.910398 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.911260 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.911423 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.916727 (+ 5304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.916735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.916740 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.916812 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.920187 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.925244 (+ 5057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.925251 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.925256 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.925423 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.925876 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.925883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.925889 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.926220 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.926360 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.926925 (+ 565us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.927154 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002}
I20260926 19:33:49.927522 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.896284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.0.0.1:43386)
0926 19:33:49.908138 (+ 11854us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.908164 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.908653 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.909386 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.909390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.910093 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.910474 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.910487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.913360 (+ 2873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.913366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.915419 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.915431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.916645 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.916653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.916850 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.917491 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.917518 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.918654 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.923191 (+ 4537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.923196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.923204 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.924077 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.924550 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.924555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.924563 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.924696 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.925492 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.925500 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.925739 (+ 239us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.926799 (+ 1060us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.927069 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11698,"spinlock_wait_cycles":37760,"thread_start_us":11699,"threads_started":1}
I20260926 19:33:49.929003 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.895586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55150)
0926 19:33:49.896319 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.896343 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.896496 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.902767 (+ 6271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.902776 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.903691 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.904184 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.904204 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.908322 (+ 4118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.908330 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.910183 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.910198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.911792 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.911800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.912205 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.913284 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.913312 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.914534 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.921109 (+ 6575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.921114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.921118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.925120 (+ 4002us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.925562 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.925569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.925572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.925743 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.926435 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.926439 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.926644 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.928272 (+ 1628us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.928595 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":1240,"threads_started":1}
I20260926 19:33:49.936425 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.897765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.6.215.253:33967)
0926 19:33:49.897954 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.897963 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.913130 (+ 15167us) server_negotiation.cc:407] Connection header received
0926 19:33:49.915248 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.915255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.915337 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.915418 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.917893 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.918927 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.921388 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.922477 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.922579 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.929269 (+ 6690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.929276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.929280 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.929320 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.932711 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.933727 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.933735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.933741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.933888 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.934351 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.934356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.934359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.934687 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.934815 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.935404 (+ 589us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.935643 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 19:33:49.937441 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.897319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:49032)
0926 19:33:49.913022 (+ 15703us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.913047 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.914704 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.915641 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.915645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.916301 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.917683 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.917696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.919098 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.919105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.920967 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.920988 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.922891 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.922898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.923063 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.923717 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.923738 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.924985 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.932913 (+ 7928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.932917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.932920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.933587 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.934036 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.934040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.934043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.934207 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.934897 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.934901 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.935110 (+ 209us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.936667 (+ 1557us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.936966 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15564,"mutex_wait_us":1,"thread_start_us":15770,"threads_started":1}
Sep 26 19:33:49 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.254@KRBTEST.COM for oryx/127.6.215.252@KRBTEST.COM
I20260926 19:33:49.949213 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.935575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.6.215.252:34391)
0926 19:33:49.935721 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:49.935727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:49.935742 (+ 15us) server_negotiation.cc:407] Connection header received
0926 19:33:49.935824 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:49.935829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:49.936342 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:49.937318 (+ 976us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:49.937506 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.938381 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.939713 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.940187 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.940333 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.943299 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:49.943311 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:49.943315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:49.943348 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:49.946362 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.947069 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.947073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.947075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.947213 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:49.947636 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:49.947642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:49.947645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:49.947954 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:49.948131 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:49.948630 (+ 499us) server_negotiation.cc:299] Negotiation successful
0926 19:33:49.948871 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:33:49.949204 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:49.934879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.0.0.1:43394)
0926 19:33:49.935256 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:49.935277 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:49.935370 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:49.936510 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:49.936516 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:49.936969 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:49.937180 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.937192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.938568 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.938574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.939564 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:49.939575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:49.940351 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:49.940357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:49.940516 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:49.941194 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:49.941210 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:49.943128 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:49.946558 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.946568 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.946570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.946896 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.947336 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:49.947348 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:49.947350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:49.947528 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:49.948137 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:49.948145 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:49.948336 (+ 191us) client_negotiation.cc:769] Sending connection context
0926 19:33:49.948492 (+ 156us) client_negotiation.cc:240] Negotiation successful
0926 19:33:49.948836 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260926 19:33:49.951284 13150 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.215.252:34391: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209"
member_type: NON_VOTER
last_known_addr {
host: "127.6.215.252"
port: 34391
}
attrs {
promote: true
}
}
dest_uuid: "cd34298261224bda8de2cb4ffa00ce2f"
cas_config_opid_index: 5
I20260926 19:33:49.952148 13150 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } }
I20260926 19:33:49.953866 13710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3f9ef2395678483bb14fe6b9aa5ec209 (127.6.215.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:49.954480 13760 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 LEARNER]: Refusing update from remote peer cd34298261224bda8de2cb4ffa00ce2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 19:33:49.955920 13842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:49.958189 13710 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.215.252:34391
I20260926 19:33:49.966293 13897 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 19:33:49.975557 13897 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:49.975883 13897 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:49.976531 13760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3f9ef2395678483bb14fe6b9aa5ec209 (127.6.215.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:49.979676 13788 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: NON_VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: true } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:49.979971 13788 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: LEARNER
I20260926 19:33:49.980531 13710 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: attempting to promote NON_VOTER d44b3986271c40009d8c47e02ff35329 to VOTER
I20260926 19:33:49.981266 13710 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } }
I20260926 19:33:49.983582 13760 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 LEARNER]: Refusing update from remote peer cd34298261224bda8de2cb4ffa00ce2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 19:33:49.984117 13896 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 19:33:49.985106 13713 master.cc:537] Master@127.6.215.252:34391 shutting down...
I20260926 19:33:49.985383 13842 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Refusing update from remote peer cd34298261224bda8de2cb4ffa00ce2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 19:33:49.985953 13841 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Refusing update from remote peer cd34298261224bda8de2cb4ffa00ce2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 19:33:49.988559 13710 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d44b3986271c40009d8c47e02ff35329 (127.6.215.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:49.990783 13897 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:49.994170 13710 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:49.994406 13710 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:49.994956 13896 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:49.995209 13896 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:49.995886 13760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d44b3986271c40009d8c47e02ff35329 (127.6.215.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
W20260926 19:33:49.996284 13821 tablet.cc:2369] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:50.000633 13788 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:50.000895 13788 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:33:50.004232 13713 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:50.004654 13713 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:50.004882 13713 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: stopping tablet replica
I20260926 19:33:50.011444 13713 master.cc:552] Master@127.6.215.252:34391 shutdown complete.
I20260926 19:33:50.011523 13713 master_runner.cc:305] Clearing existing system tablet
I20260926 19:33:50.011920 13713 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:50.012025 13713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260926 19:33:50.015264 13136 consensus_peers.cc:487] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f -> Peer 3f9ef2395678483bb14fe6b9aa5ec209 (127.6.215.252:34391): Couldn't send request to peer 3f9ef2395678483bb14fe6b9aa5ec209. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260926 19:33:50.016006 13713 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:33:50.017642 13902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:50.018016 13713 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:50.018435 13713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 19:33:50.018577 13713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
uuid: "3f9ef2395678483bb14fe6b9aa5ec209"
format_stamp: "Formatted at 2026-09-26 19:33:49 on dist-test-slave-phbh"
server_key: "d5240700546c6039e94e77dc4d357908"
server_key_iv: "1795778c80e9de4cd646d76167bf9b87"
server_key_version: "encryptionkey@0"
I20260926 19:33:50.023711 13713 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:50.030699 13713 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 19:33:50.030973 13713 log.cc:1196] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 19:33:50.031526 13713 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Deleting consensus metadata
I20260926 19:33:50.031821 13713 master_runner.cc:315] Copying system tablet from 127.6.215.254:45377
I20260926 19:33:50.034350 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 13211
I20260926 19:33:50.041729 13713 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: tablet copy: Beginning tablet copy session from remote peer at address 127.6.215.254:45377
I20260926 19:33:50.053869 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:34191
--local_ip_for_outbound_sockets=127.6.215.193
--tserver_master_addrs=127.6.215.254:45377
--webserver_port=38325
--webserver_interface=127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab
--principal=oryx/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:33:50.057410 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.042685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55152 (local address 127.6.215.254:45377)
0926 19:33:50.042934 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:50.042942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:50.043530 (+ 588us) server_negotiation.cc:407] Connection header received
0926 19:33:50.043796 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:50.043803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:50.043891 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:50.043961 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:50.045158 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.046288 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.048217 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.048933 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.049058 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.051652 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:50.051659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:50.051664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:50.051706 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:50.054822 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.055449 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.055454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.055464 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.055586 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.055970 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.055977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.055980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.056235 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:50.056433 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:50.056817 (+ 384us) server_negotiation.cc:299] Negotiation successful
0926 19:33:50.057037 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260926 19:33:50.057765 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.042893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55152)
0926 19:33:50.043417 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:50.043443 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:50.043645 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:50.044159 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:50.044164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:50.044703 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:50.044984 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.044997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.046489 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.046494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.048055 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.048069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.049117 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.049127 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.049364 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.050157 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:50.050184 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:50.051412 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:50.054975 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.054980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.054984 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.055321 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.055706 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.055710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.055716 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.055856 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.056362 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:50.056367 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:50.056536 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 19:33:50.056676 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 19:33:50.057083 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":266,"threads_started":1}
I20260926 19:33:50.058488 13170 tablet_copy_service.cc:140] P cd34298261224bda8de2cb4ffa00ce2f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3f9ef2395678483bb14fe6b9aa5ec209 ({username='oryx', principal='oryx/127.6.215.252@KRBTEST.COM'} at 127.0.0.1:55152)
I20260926 19:33:50.058652 13170 tablet_copy_service.cc:161] P cd34298261224bda8de2cb4ffa00ce2f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3f9ef2395678483bb14fe6b9aa5ec209 at {username='oryx', principal='oryx/127.6.215.252@KRBTEST.COM'} at 127.0.0.1:55152: session id = 3f9ef2395678483bb14fe6b9aa5ec209-00000000000000000000000000000000
I20260926 19:33:50.061700 13170 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 19:33:50.063616 13713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 19:33:50.070495 13713 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: tablet copy: Starting download of 0 data blocks...
I20260926 19:33:50.070798 13713 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: tablet copy: Starting download of 1 WAL segments...
I20260926 19:33:50.073316 13713 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 19:33:50.078576 13170 tablet_copy_service.cc:342] P cd34298261224bda8de2cb4ffa00ce2f: Request end of tablet copy session 3f9ef2395678483bb14fe6b9aa5ec209-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.215.252@KRBTEST.COM'} at 127.0.0.1:55152
I20260926 19:33:50.078722 13170 tablet_copy_service.cc:433] P cd34298261224bda8de2cb4ffa00ce2f: ending tablet copy session 3f9ef2395678483bb14fe6b9aa5ec209-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3f9ef2395678483bb14fe6b9aa5ec209
I20260926 19:33:50.089813 13713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:50.092324 13910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:50.098723 13911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:50.106463 13914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:50.114374 13713 server_base.cc:1034] running on GCE node
I20260926 19:33:50.114758 13713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:50.115448 13713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:50.116654 13713 hybrid_clock.cc:648] HybridClock initialized: now 1790451230116636 us; error 58 us; skew 500 ppm
I20260926 19:33:50.116933 13713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:50 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451230, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:50.119299 13713 init.cc:377] Logged in from keytab as oryx/127.6.215.252@KRBTEST.COM (short username oryx)
I20260926 19:33:50.124445 13713 webserver.cc:460] Webserver started at http://127.6.215.252:42777/ using document root <none> and password file <none>
I20260926 19:33:50.124785 13713 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:50.124907 13713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:50.128046 13713 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260926 19:33:50.130018 13920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:50.130558 13713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 19:33:50.130677 13713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
uuid: "3f9ef2395678483bb14fe6b9aa5ec209"
format_stamp: "Formatted at 2026-09-26 19:33:49 on dist-test-slave-phbh"
server_key: "d5240700546c6039e94e77dc4d357908"
server_key_iv: "1795778c80e9de4cd646d76167bf9b87"
server_key_version: "encryptionkey@0"
I20260926 19:33:50.130861 13713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:50.152897 13713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:50.156973 13713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:50.179473 13713 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.252:34391
I20260926 19:33:50.179527 13971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.252:34391 every 8 connection(s)
I20260926 19:33:50.180454 13713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
I20260926 19:33:50.189414 13972 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:33:50.193210 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Bootstrap starting.
I20260926 19:33:50.211838 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:33:50.212476 13972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Bootstrap complete.
I20260926 19:33:50.213596 13972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } }
I20260926 19:33:50.213897 13972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3f9ef2395678483bb14fe6b9aa5ec209, State: Initialized, Role: LEARNER
I20260926 19:33:50.214159 13972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } }
I20260926 19:33:50.215103 13975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:50.215540 13975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: This master's current role is: LEARNER
I20260926 19:33:50.215598 13972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:50.223565 13986 catalog_manager.cc:1259] Loaded cluster ID: c865ba0a47534565a1cbb132a5047d56
I20260926 19:33:50.223673 13986 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: loading cluster ID for follower catalog manager: success
I20260926 19:33:50.227306 13986 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: acquiring CA information for follower catalog manager: success
I20260926 19:33:50.229753 13986 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 19:33:50.430212 13909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:50.430533 13909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:50.430732 13909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:50.438874 13909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:50.439019 13909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:50.439203 13909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:50.448848 13909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab
--principal=oryx/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:34191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=38325
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:50.450212 13909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:50.451522 13909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:50.465734 13994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:50.465765 13991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:50.465893 13992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:50.470446 13909 server_base.cc:1034] running on GCE node
I20260926 19:33:50.471099 13909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:50.472091 13909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:50.473315 13909 hybrid_clock.cc:648] HybridClock initialized: now 1790451230473262 us; error 50 us; skew 500 ppm
I20260926 19:33:50.473635 13909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:50 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451230, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:50.477133 13909 init.cc:377] Logged in from keytab as oryx/127.6.215.193@KRBTEST.COM (short username oryx)
I20260926 19:33:50.478569 13909 webserver.cc:460] Webserver started at http://127.6.215.193:38325/ using document root <none> and password file <none>
I20260926 19:33:50.479290 13909 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:50.479449 13909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:50.485417 13909 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 19:33:50.488384 14001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:50.489980 13909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:50.490161 13909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "f948ce0c2b6149528c17d39f278cfe76"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "8e5850da3073e7d2cd5d192feddb8439"
server_key_iv: "e9c9a856ca1f28b8058ebfcd1aa280b9"
server_key_version: "encryptionkey@0"
I20260926 19:33:50.491129 13909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:50.519411 14006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.512506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.0.0.1:43400)
0926 19:33:50.513140 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:50.513159 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:50.513301 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:50.513756 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:50.513761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:50.513781 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:50.514106 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.514119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.515471 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.515477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.517297 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.517311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.518582 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.518588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.518773 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.518807 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 19:33:50.518899 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 19:33:50.519060 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":428,"threads_started":1}
I20260926 19:33:50.519428 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.512669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.6.215.252:34391)
0926 19:33:50.513462 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:50.513467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:50.513483 (+ 16us) server_negotiation.cc:407] Connection header received
0926 19:33:50.513543 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:50.513547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:50.513615 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:50.513718 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:50.514275 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.515332 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.517506 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.518411 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.518576 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.518779 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:50.518956 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 19:33:50.519129 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":561,"threads_started":1}
I20260926 19:33:50.521063 13947 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260926 19:33:50.521831 13975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd34298261224bda8de2cb4ffa00ce2f. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:50.522092 13975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: This master's current role is: LEARNER
I20260926 19:33:50.524051 13909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:50.528206 13909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:50.528446 13909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:50.529405 13909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:50.530752 13909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:50.530838 13909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:50.530905 13909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:50.530999 13909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:50.566972 13909 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:34191
I20260926 19:33:50.567054 14115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:34191 every 8 connection(s)
I20260926 19:33:50.568463 13909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:50.578562 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 13909
I20260926 19:33:50.578886 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 13345
Sep 26 19:33:50 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451230, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.252@KRBTEST.COM
Sep 26 19:33:50 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451230, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
Sep 26 19:33:50 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451230, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.253@KRBTEST.COM
I20260926 19:33:50.598091 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.571923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:42639 (local address 127.6.215.252:34391)
0926 19:33:50.572125 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:50.572131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:50.573638 (+ 1507us) server_negotiation.cc:407] Connection header received
0926 19:33:50.575926 (+ 2288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:50.575933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:50.576018 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:50.576072 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:50.578189 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.579719 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.582577 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.583224 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.583326 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.588665 (+ 5339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:50.588673 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:50.588678 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:50.588722 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:50.593201 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.594642 (+ 1441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.594648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.594651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.594817 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.595342 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.595348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.595352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.595720 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:50.595865 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:50.597488 (+ 1623us) server_negotiation.cc:299] Negotiation successful
0926 19:33:50.597721 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 19:33:50.600658 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.572483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.6.215.193:42639)
0926 19:33:50.573474 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:50.573522 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:50.575669 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:50.576347 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:50.576356 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:50.577278 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:50.578013 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.578031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.579899 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.579917 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.582409 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.582422 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.583648 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.583655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.584079 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.585246 (+ 1167us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:50.585277 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:50.588383 (+ 3106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:50.593797 (+ 5414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.593806 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.593825 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.594498 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.594968 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.594972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.594976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.595199 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.596518 (+ 1319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:50.596526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:50.597005 (+ 479us) client_negotiation.cc:769] Sending connection context
0926 19:33:50.598428 (+ 1423us) client_negotiation.cc:240] Negotiation successful
0926 19:33:50.599027 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":423,"spinlock_wait_cycles":35584,"thread_start_us":449,"threads_started":1}
I20260926 19:33:50.600828 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.574216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:60691 (local address 127.6.215.254:45377)
0926 19:33:50.574807 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:50.574814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:50.574834 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:33:50.576333 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:50.576339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:50.576411 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:50.576522 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:50.578189 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.579733 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.582581 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.583224 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.583337 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.590294 (+ 6957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:50.590301 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:50.590305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:50.590349 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:50.595720 (+ 5371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.596810 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.596816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.596819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.596977 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.598476 (+ 1499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.598481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.598485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.599360 (+ 875us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:50.599522 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:50.600153 (+ 631us) server_negotiation.cc:299] Negotiation successful
0926 19:33:50.600364 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":358,"threads_started":1}
I20260926 19:33:50.600632 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.574077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.193:60691)
0926 19:33:50.574679 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:50.574698 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:50.575669 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:50.576569 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:50.576573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:50.577324 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:50.578038 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.578046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.579899 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.579904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.582399 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.582422 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.583648 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.583655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.584081 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.586766 (+ 2685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:50.587386 (+ 620us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:50.590046 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:50.595833 (+ 5787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.595840 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.595846 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.596368 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.597114 (+ 746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.597120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.597123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.597301 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.599471 (+ 2170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:50.599479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:50.599659 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 19:33:50.599780 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 19:33:50.600030 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":227,"spinlock_wait_cycles":5120,"thread_start_us":278,"threads_started":1}
I20260926 19:33:50.604252 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.571763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:38163 (local address 127.6.215.253:33967)
0926 19:33:50.572678 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:50.572685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:50.573632 (+ 947us) server_negotiation.cc:407] Connection header received
0926 19:33:50.575906 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:50.575912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:50.576021 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:50.576140 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:50.578158 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.579269 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.586670 (+ 7401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.587585 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.587746 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.593331 (+ 5585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:50.593338 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:50.593342 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:50.593383 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:50.598978 (+ 5595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.599769 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.599774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.599777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.599908 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.600367 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.600372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.600375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.600759 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:50.601335 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:50.603124 (+ 1789us) server_negotiation.cc:299] Negotiation successful
0926 19:33:50.603386 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":756,"threads_started":1}
I20260926 19:33:50.604503 14117 heartbeater.cc:347] Connected to a master server at 127.6.215.252:34391
I20260926 19:33:50.604846 14116 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45377
I20260926 19:33:50.605057 14116 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:50.606031 14116 heartbeater.cc:511] Master 127.6.215.254:45377 requested a full tablet report, sending...
I20260926 19:33:50.606964 14117 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:50.607394 14117 heartbeater.cc:511] Master 127.6.215.252:34391 requested a full tablet report, sending...
Sep 26 19:33:50 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.194@KRBTEST.COM
I20260926 19:33:50.607676 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.572483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.6.215.193:38163)
0926 19:33:50.573474 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:50.573522 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:50.575649 (+ 2127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:50.576348 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:50.576357 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:50.577227 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:50.577916 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.577936 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.584872 (+ 6936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.584879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.586488 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:50.586505 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.588782 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.588791 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:50.589026 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.590164 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:50.590194 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:50.593090 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:50.599120 (+ 6030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.599125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.599131 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.599602 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.600035 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:50.600039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:50.600043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:50.600229 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:50.601399 (+ 1170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:50.601415 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:50.601679 (+ 264us) client_negotiation.cc:769] Sending connection context
0926 19:33:50.601817 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 19:33:50.602109 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":27,"thread_start_us":425,"threads_started":1}
I20260926 19:33:50.608614 13150 ts_manager.cc:194] Re-registered known tserver with Master: f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191)
I20260926 19:33:50.609275 13937 ts_manager.cc:194] Registered new tserver with Master: f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191)
I20260926 19:33:50.609583 13150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:60691
I20260926 19:33:50.610028 14118 heartbeater.cc:347] Connected to a master server at 127.6.215.253:33967
I20260926 19:33:50.610165 14118 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:50.610630 14118 heartbeater.cc:511] Master 127.6.215.253:33967 requested a full tablet report, sending...
I20260926 19:33:50.612072 13750 ts_manager.cc:194] Registered new tserver with Master: f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191)
I20260926 19:33:50.618636 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:50.579864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.6.215.194:45579)
0926 19:33:50.592046 (+ 12182us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:50.592058 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:50.592089 (+ 31us) server_negotiation.cc:407] Connection header received
0926 19:33:50.592166 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:50.592176 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:50.592486 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:50.592651 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:50.595039 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.596415 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.598139 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:50.599100 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:50.599241 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:50.612152 (+ 12911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:50.612170 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:50.612186 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:50.612230 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:50.615399 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.616125 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.616131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.616134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.616736 (+ 602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:50.617116 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:50.617119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:50.617121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:50.617503 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:50.617782 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:50.618035 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 19:33:50.618298 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11974,"thread_start_us":12396,"threads_started":1}
I20260926 19:33:50.619864 13468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:50.702160 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 13345
I20260926 19:33:50.719528 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:45579
--local_ip_for_outbound_sockets=127.6.215.194
--tserver_master_addrs=127.6.215.254:45377
--webserver_port=37433
--webserver_interface=127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab
--principal=oryx/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:51.048347 14128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:51.048653 14128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:51.048751 14128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:51.056243 14128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:51.056372 14128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:51.056488 14128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:51.058997 13897 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: NON_VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: true } } }
I20260926 19:33:51.059332 13897 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:51.061060 13897 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: attempting to promote NON_VOTER 3f9ef2395678483bb14fe6b9aa5ec209 to VOTER
I20260926 19:33:51.061753 13897 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } }
I20260926 19:33:51.063949 13947 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 LEARNER]: Refusing update from remote peer cd34298261224bda8de2cb4ffa00ce2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 19:33:51.064037 13760 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Refusing update from remote peer cd34298261224bda8de2cb4ffa00ce2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 19:33:51.064572 14133 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 19:33:51.065006 13897 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 19:33:51.068619 14133 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3f9ef2395678483bb14fe6b9aa5ec209 (127.6.215.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.069537 13947 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3f9ef2395678483bb14fe6b9aa5ec209 (127.6.215.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.070240 14128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab
--principal=oryx/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:45579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=37433
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:51.071466 14128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:51.072408 14133 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.072757 14133 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:51.072544 14132 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.072917 14132 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:51.072803 13760 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3f9ef2395678483bb14fe6b9aa5ec209 (127.6.215.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.073083 14133 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.073280 14128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 19:33:51.073321 14133 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:51.076046 14139 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.076752 14139 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:33:51.076651 14138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd34298261224bda8de2cb4ffa00ce2f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:51.076987 14138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 19:33:51.087790 14141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:51.087857 14142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:51.088114 14144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:51.088974 14128 server_base.cc:1034] running on GCE node
I20260926 19:33:51.089596 14128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:51.090551 14128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:51.091754 14128 hybrid_clock.cc:648] HybridClock initialized: now 1790451231091703 us; error 39 us; skew 500 ppm
I20260926 19:33:51.092093 14128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:51.095552 14128 init.cc:377] Logged in from keytab as oryx/127.6.215.194@KRBTEST.COM (short username oryx)
I20260926 19:33:51.097019 14128 webserver.cc:460] Webserver started at http://127.6.215.194:37433/ using document root <none> and password file <none>
I20260926 19:33:51.097815 14128 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:51.097939 14128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:51.104110 14128 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260926 19:33:51.107108 14151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:51.108611 14128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 19:33:51.108922 14128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "07f3f61459da425bb46983f15833d8a4"
format_stamp: "Formatted at 2026-09-26 19:33:47 on dist-test-slave-phbh"
server_key: "d090557af7819301fdaabfa838c3df25"
server_key_iv: "1c035d0ad6cb05920a682b04b1927c03"
server_key_version: "encryptionkey@0"
I20260926 19:33:51.109913 14128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:51.131937 14128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:51.136428 14128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:51.136734 14128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:51.137774 14128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:51.139196 14128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:51.139326 14128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:51.139461 14128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:51.139546 14128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:51.171336 14128 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:45579
I20260926 19:33:51.171392 14263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:45579 every 8 connection(s)
I20260926 19:33:51.172950 14128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:51.180658 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 14128
I20260926 19:33:51.181030 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 13478
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.195@KRBTEST.COM
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.252@KRBTEST.COM
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.253@KRBTEST.COM
I20260926 19:33:51.229550 14273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.182134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.6.215.195:36411)
0926 19:33:51.197347 (+ 15213us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.197360 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.197399 (+ 39us) server_negotiation.cc:407] Connection header received
0926 19:33:51.197497 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.197509 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.197831 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.198039 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.204448 (+ 6409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.209599 (+ 5151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.211963 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.213235 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.213438 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.220863 (+ 7425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.220880 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.220897 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.220939 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.225273 (+ 4334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.226749 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.226758 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.226761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.226918 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.227396 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.227404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.227410 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.227839 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.228108 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.228753 (+ 645us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.229050 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15034,"thread_start_us":14888,"threads_started":1}
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:51.232666 13601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:51.237562 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.202962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:44035 (local address 127.6.215.254:45377)
0926 19:33:51.215399 (+ 12437us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.215405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.215978 (+ 573us) server_negotiation.cc:407] Connection header received
0926 19:33:51.216760 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.216790 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.216899 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.217047 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.219074 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.220408 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.223453 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.224118 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.224224 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.230714 (+ 6490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.230720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.230725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.230776 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.233883 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.235026 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.235034 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.235040 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.235225 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.235648 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.235653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.235657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.236043 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.236216 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.236859 (+ 643us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.237136 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12302,"thread_start_us":12166,"threads_started":1}
I20260926 19:33:51.237776 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.175761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:48431 (local address 127.6.215.252:34391)
0926 19:33:51.215197 (+ 39436us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.215203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.215224 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:33:51.216008 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.216013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.216096 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.216200 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.218988 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.220132 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.223456 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.224096 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.224289 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.229480 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.229487 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.229492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.229533 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.233154 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.234312 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.234317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.234321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.234902 (+ 581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.235433 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.235437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.235443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.235783 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.235935 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.237215 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.237426 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39289,"thread_start_us":39097,"threads_started":1}
I20260926 19:33:51.238796 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.210014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:44035)
0926 19:33:51.215834 (+ 5820us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.215862 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.216570 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.217237 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.217247 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.217988 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:51.218778 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.218820 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.220575 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.220581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.223281 (+ 2700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.223297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.225013 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.225021 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.225256 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.226087 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:51.226113 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:51.230492 (+ 4379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:51.234025 (+ 3533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.234032 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.234036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.234893 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.235364 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.235377 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.235383 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.235548 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.236190 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:51.236194 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:51.236369 (+ 175us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.236692 (+ 323us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.237295 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5666,"mutex_wait_us":338,"thread_start_us":5542,"threads_started":1}
I20260926 19:33:51.238827 14271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.177180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.6.215.194:48431)
0926 19:33:51.202425 (+ 25245us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.202444 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.208060 (+ 5616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.216418 (+ 8358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.216421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.217630 (+ 1209us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:51.218777 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.218804 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.220344 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.220349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.223254 (+ 2905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.223269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.224589 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.224596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.225080 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.225909 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:51.225970 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:51.229159 (+ 3189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:51.233352 (+ 4193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.233364 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.233388 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.234149 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.235059 (+ 910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.235064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.235068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.235266 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.235909 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:51.235917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:51.236371 (+ 454us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.236692 (+ 321us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.238112 (+ 1420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25130,"mutex_wait_us":1520,"spinlock_wait_cycles":56320,"thread_start_us":31116,"threads_started":1}
I20260926 19:33:51.240262 14264 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45377
I20260926 19:33:51.240284 14265 heartbeater.cc:347] Connected to a master server at 127.6.215.252:34391
I20260926 19:33:51.240554 14264 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:51.240561 14265 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:51.241546 14265 heartbeater.cc:511] Master 127.6.215.252:34391 requested a full tablet report, sending...
I20260926 19:33:51.241581 14264 heartbeater.cc:511] Master 127.6.215.254:45377 requested a full tablet report, sending...
I20260926 19:33:51.242599 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.176518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.6.215.194:44037)
0926 19:33:51.201384 (+ 24866us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.201447 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.208449 (+ 7002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.216197 (+ 7748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.216210 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.217630 (+ 1420us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:51.218777 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.218804 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.220344 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.220349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.223286 (+ 2937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.223295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.224628 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.224636 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.225043 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.225909 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:51.225971 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:51.230418 (+ 4447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:51.238565 (+ 8147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.238573 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.238579 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.240090 (+ 1511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.240526 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.240532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.240541 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.240714 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.241457 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:51.241461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:51.241703 (+ 242us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.241892 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.242157 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24533,"mutex_wait_us":565,"spinlock_wait_cycles":40832,"thread_start_us":24259,"threads_started":1}
I20260926 19:33:51.243247 13150 ts_manager.cc:194] Re-registered known tserver with Master: 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579)
I20260926 19:33:51.243417 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.175661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:44037 (local address 127.6.215.253:33967)
0926 19:33:51.215512 (+ 39851us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.215519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.215543 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:33:51.215602 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.215608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.215694 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.215851 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.218988 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.220132 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.223450 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.224159 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.224269 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.230856 (+ 6587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.230864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.230869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.230911 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.238407 (+ 7496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.240255 (+ 1848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.240261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.240265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.240424 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.240872 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.240877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.240880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.241231 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.241392 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.243026 (+ 1634us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.243192 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38996,"thread_start_us":39651,"threads_started":1}
I20260926 19:33:51.243428 13937 ts_manager.cc:194] Registered new tserver with Master: 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579)
I20260926 19:33:51.244617 13150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:44035
I20260926 19:33:51.246526 14266 heartbeater.cc:347] Connected to a master server at 127.6.215.253:33967
I20260926 19:33:51.246629 14266 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:51.247053 14266 heartbeater.cc:511] Master 127.6.215.253:33967 requested a full tablet report, sending...
I20260926 19:33:51.248072 13750 ts_manager.cc:194] Registered new tserver with Master: 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579)
I20260926 19:33:51.324788 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 13478
I20260926 19:33:51.340664 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:36411
--local_ip_for_outbound_sockets=127.6.215.195
--tserver_master_addrs=127.6.215.254:45377
--webserver_port=41269
--webserver_interface=127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab
--principal=oryx/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:33:51.612478 14116 heartbeater.cc:503] Master 127.6.215.254:45377 was elected leader, sending a full tablet report...
W20260926 19:33:51.673738 14277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:51.674063 14277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:51.674230 14277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:51.681476 14277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:51.681625 14277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:51.681802 14277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:51.692093 14277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab
--principal=oryx/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:36411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=41269
--tserver_master_addrs=127.6.215.254:45377,127.6.215.253:33967,127.6.215.252:34391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:51.693614 14277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:51.694926 14277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:51.708504 14282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:51.708472 14283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:51.708606 14285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:51.709491 14277 server_base.cc:1034] running on GCE node
I20260926 19:33:51.710116 14277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:51.711086 14277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:51.712324 14277 hybrid_clock.cc:648] HybridClock initialized: now 1790451231712239 us; error 64 us; skew 500 ppm
I20260926 19:33:51.712637 14277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:51.716104 14277 init.cc:377] Logged in from keytab as oryx/127.6.215.195@KRBTEST.COM (short username oryx)
I20260926 19:33:51.717625 14277 webserver.cc:460] Webserver started at http://127.6.215.195:41269/ using document root <none> and password file <none>
I20260926 19:33:51.718423 14277 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:51.718571 14277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:51.723882 14277 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 19:33:51.726722 14292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:51.728015 14277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:33:51.728185 14277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "361e1a07f3014ab2b12ce6ff07fc3164"
format_stamp: "Formatted at 2026-09-26 19:33:48 on dist-test-slave-phbh"
server_key: "76a8b1629b9c34fe7eaaa1573b7e61d5"
server_key_iv: "95831cf6691b4b9c07cd65e06db72f02"
server_key_version: "encryptionkey@0"
I20260926 19:33:51.729206 14277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:51.746224 14277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:51.750285 14277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:51.750545 14277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:51.751457 14277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:33:51.752722 14277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:51.752868 14277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:51.752998 14277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:51.753085 14277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:51.787456 14277 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:36411
I20260926 19:33:51.787519 14404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:36411 every 8 connection(s)
I20260926 19:33:51.788892 14277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:51.791066 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 14277
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.253@KRBTEST.COM
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.252@KRBTEST.COM
Sep 26 19:33:51 dist-test-slave-phbh krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451231, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:51.831046 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.791963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:38449 (local address 127.6.215.253:33967)
0926 19:33:51.807605 (+ 15642us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.807611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.807631 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:33:51.807687 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.807692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.807766 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.807884 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.808758 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.809885 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.811626 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.812246 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.812357 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.816903 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.816911 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.816929 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.816972 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.821646 (+ 4674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.826694 (+ 5048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.826700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.826704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.826859 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.829244 (+ 2385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.829252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.829258 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.829622 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.829772 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.830400 (+ 628us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.830619 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15516,"thread_start_us":16027,"threads_started":1}
I20260926 19:33:51.832041 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.801313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:56063 (local address 127.6.215.254:45377)
0926 19:33:51.812029 (+ 10716us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.812037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.812061 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:33:51.812127 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.812135 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.812227 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.812903 (+ 676us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.814603 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.815938 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.818217 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.818975 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.819100 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.823795 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.823805 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.823816 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.823858 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.827515 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.828253 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.828261 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.828267 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.828431 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.828872 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.828877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.828880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.829223 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.829355 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.831506 (+ 2151us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.831724 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10562,"thread_start_us":11273,"threads_started":1}
I20260926 19:33:51.832409 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.792800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.6.215.195:38449)
0926 19:33:51.797975 (+ 5175us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.797992 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.801722 (+ 3730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.807902 (+ 6180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.807906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.808412 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:51.808604 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.808612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.810050 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.810056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.811493 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.811502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.812406 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.812412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.812585 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.813350 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:51.813364 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:51.816616 (+ 3252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:51.821811 (+ 5195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.821824 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.821845 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.822781 (+ 936us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.827021 (+ 4240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.827026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.827030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.827283 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.829841 (+ 2558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:51.829845 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:51.829989 (+ 144us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.830140 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.831445 (+ 1305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4997,"mutex_wait_us":1565,"spinlock_wait_cycles":356608,"thread_start_us":4262,"threads_started":1}
I20260926 19:33:51.832854 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.801345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.195:56063)
0926 19:33:51.802178 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.802200 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.802309 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.813117 (+ 10808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.813125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.813915 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:51.814338 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.814365 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.816108 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.816112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.818038 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.818054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.819345 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.819350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.819567 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.820155 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:51.820178 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:51.823564 (+ 3386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:51.827657 (+ 4093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.827661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.827666 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.828105 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.828559 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.828563 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.828566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.828716 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.829412 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:51.829417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:51.829559 (+ 142us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.829724 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.830302 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":598,"threads_started":1}
I20260926 19:33:51.834010 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.793522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:41087 (local address 127.6.215.252:34391)
0926 19:33:51.802441 (+ 8919us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.802447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.802473 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:33:51.802521 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.802524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.802593 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.802779 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.805029 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.806458 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.810144 (+ 3686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.810764 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.810862 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.821940 (+ 11078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.821948 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.821953 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.821996 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.825504 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.826380 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.826385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.826389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.826549 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.827729 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.827735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.827738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.828119 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.828247 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.833402 (+ 5155us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.833637 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8783,"thread_start_us":8760,"threads_started":1}
I20260926 19:33:51.834461 14405 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45377
I20260926 19:33:51.834776 14405 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:51.835381 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.792797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.6.215.195:41087)
0926 19:33:51.794353 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.794408 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.801302 (+ 6894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.803032 (+ 1730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.803047 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.803969 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:51.804727 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.804758 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.806754 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.806762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.809982 (+ 3220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.809997 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.811215 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.811221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.811892 (+ 671us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.813007 (+ 1115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:51.813042 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:51.820743 (+ 7701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:51.825700 (+ 4957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.825708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.825715 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.826230 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.827429 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:51.827434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:51.827439 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:51.827605 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:51.828333 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:51.828345 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:51.828871 (+ 526us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.834633 (+ 5762us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.834951 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1253,"mutex_wait_us":1,"thread_start_us":980,"threads_started":1}
I20260926 19:33:51.835668 14405 heartbeater.cc:511] Master 127.6.215.254:45377 requested a full tablet report, sending...
I20260926 19:33:51.837164 13150 ts_manager.cc:194] Re-registered known tserver with Master: 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:51.838294 13150 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:56063
I20260926 19:33:51.840179 14406 heartbeater.cc:347] Connected to a master server at 127.6.215.252:34391
I20260926 19:33:51.840303 14406 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:51.840734 14406 heartbeater.cc:511] Master 127.6.215.252:34391 requested a full tablet report, sending...
I20260926 19:33:51.841955 13937 ts_manager.cc:194] Registered new tserver with Master: 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:51.846057 14407 heartbeater.cc:347] Connected to a master server at 127.6.215.253:33967
I20260926 19:33:51.846194 14407 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:51.846634 14407 heartbeater.cc:511] Master 127.6.215.253:33967 requested a full tablet report, sending...
I20260926 19:33:51.847805 13750 ts_manager.cc:194] Registered new tserver with Master: 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:51.854866 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.829508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49042 (local address 127.6.215.253:33967)
0926 19:33:51.831271 (+ 1763us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.831277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.837662 (+ 6385us) server_negotiation.cc:407] Connection header received
0926 19:33:51.837933 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.837939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.838032 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.838095 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.841072 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.842097 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.843888 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.844491 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.844605 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.848831 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.848839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.848845 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.848895 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.852017 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.852883 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.852891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.852894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.853034 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.853405 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.853410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.853414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.853773 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.854026 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.854348 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.854558 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1674,"thread_start_us":15064,"threads_started":1}
I20260926 19:33:51.855558 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.806795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55164 (local address 127.6.215.254:45377)
0926 19:33:51.828616 (+ 21821us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.828624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.828646 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:33:51.828704 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.828712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.829590 (+ 878us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.829692 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.832993 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.834383 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.843026 (+ 8643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.843882 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.844021 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.849306 (+ 5285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.849315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.849321 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.849366 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.852715 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.853553 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.853558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.853561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.853683 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.854058 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.854064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.854067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.854424 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.854813 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.854952 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.855157 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21653,"thread_start_us":28949,"threads_started":1}
I20260926 19:33:51.857583 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.817053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.6.215.252:34391)
0926 19:33:51.834230 (+ 17177us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.834236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.834255 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:33:51.834303 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.834307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.834418 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.834524 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:51.837066 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.838330 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.845178 (+ 6848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.845831 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.845949 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.850277 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:51.850284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:51.850289 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:51.850327 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:51.853670 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.855507 (+ 1837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.855515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.855520 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.855663 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:51.856080 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:51.856085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:51.856088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:51.856370 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:51.856612 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.856989 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.857201 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17085,"thread_start_us":28144,"threads_started":1}
I20260926 19:33:51.859472 13150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:51.867605 13150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:51.907569 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.886690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.6.215.195:36411)
0926 19:33:51.899032 (+ 12342us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.899039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.899065 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:33:51.899144 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.899150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.899354 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.899570 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:51.900222 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.901808 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.905080 (+ 3272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.906555 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.906699 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.906924 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.907108 (+ 184us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.907256 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12198,"thread_start_us":13054,"threads_started":1}
I20260926 19:33:51.907603 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.886301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:36411 (local address 127.0.0.1:46176)
0926 19:33:51.897379 (+ 11078us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.897396 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.898134 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.899546 (+ 1412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.899550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.899573 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:51.899960 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.899973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.902841 (+ 2868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.902854 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.904848 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.904862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.906741 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.906747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.906909 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.906960 (+ 51us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.907053 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.907183 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10938,"mutex_wait_us":133,"thread_start_us":10809,"threads_started":1}
I20260926 19:33:51.909162 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.885484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34191 (local address 127.0.0.1:43166)
0926 19:33:51.886052 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.886073 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.886192 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.901754 (+ 15562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.901759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.901783 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:51.902168 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.902183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.903826 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.903832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.906147 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.906163 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.908324 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.908330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.908474 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.908504 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.908601 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.908724 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":21,"thread_start_us":312,"threads_started":1}
I20260926 19:33:51.909189 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.886075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.6.215.193:34191)
0926 19:33:51.901133 (+ 15058us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.901142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.901172 (+ 30us) server_negotiation.cc:407] Connection header received
0926 19:33:51.901272 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.901279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.901560 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.901763 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:51.902347 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.903691 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.906316 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.908129 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.908269 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.908473 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.908655 (+ 182us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.908886 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14916,"thread_start_us":15946,"threads_started":1}
I20260926 19:33:51.910175 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.886301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:45579 (local address 127.0.0.1:49474)
0926 19:33:51.896718 (+ 10417us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.896737 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.896884 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.902537 (+ 5653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.902541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.902568 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:51.902938 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.902977 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.904498 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.904503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.906503 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.906519 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.909233 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.909239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.909565 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.909600 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.909685 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.909812 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10290,"mutex_wait_us":38,"thread_start_us":10139,"threads_started":1}
I20260926 19:33:51.910804 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.886624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.6.215.194:45579)
0926 19:33:51.901338 (+ 14714us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.901359 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.901385 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:33:51.901465 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.901471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.901662 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.902497 (+ 835us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:51.903148 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.904330 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.906677 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.908703 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.908898 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.909080 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.910499 (+ 1419us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.910627 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14310,"thread_start_us":14520,"threads_started":1}
I20260926 19:33:51.911796 14340 tablet_service.cc:1467] Processing CreateTablet for tablet a82df00947c949c19d6017a8ac874b31 (DEFAULT_TABLE table=test-table [id=5607dda2061249aa9b38e5124aeea823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:51.913349 14340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82df00947c949c19d6017a8ac874b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:51.914186 14199 tablet_service.cc:1467] Processing CreateTablet for tablet a82df00947c949c19d6017a8ac874b31 (DEFAULT_TABLE table=test-table [id=5607dda2061249aa9b38e5124aeea823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:51.915481 14199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82df00947c949c19d6017a8ac874b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:51.915685 14051 tablet_service.cc:1467] Processing CreateTablet for tablet a82df00947c949c19d6017a8ac874b31 (DEFAULT_TABLE table=test-table [id=5607dda2061249aa9b38e5124aeea823]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:51.916868 14051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82df00947c949c19d6017a8ac874b31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:51.928158 14439 tablet_bootstrap.cc:492] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Bootstrap starting.
I20260926 19:33:51.932000 14439 tablet_bootstrap.cc:654] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:51.933473 14441 tablet_bootstrap.cc:492] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Bootstrap starting.
I20260926 19:33:51.933454 14439 log.cc:824] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:51.935200 14440 tablet_bootstrap.cc:492] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Bootstrap starting.
I20260926 19:33:51.937697 14440 tablet_bootstrap.cc:654] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:51.937902 14441 tablet_bootstrap.cc:654] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:51.938658 14440 log.cc:824] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:51.939299 14441 log.cc:824] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:51.942975 14441 tablet_bootstrap.cc:492] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: No bootstrap required, opened a new log
I20260926 19:33:51.943205 14439 tablet_bootstrap.cc:492] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: No bootstrap required, opened a new log
I20260926 19:33:51.943253 14441 ts_tablet_manager.cc:1397] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260926 19:33:51.943398 14440 tablet_bootstrap.cc:492] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: No bootstrap required, opened a new log
I20260926 19:33:51.943495 14439 ts_tablet_manager.cc:1397] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260926 19:33:51.943634 14440 ts_tablet_manager.cc:1397] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 19:33:51.950412 14439 raft_consensus.cc:348] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.950775 14439 raft_consensus.cc:374] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:51.950866 14439 raft_consensus.cc:729] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f948ce0c2b6149528c17d39f278cfe76, State: Initialized, Role: FOLLOWER
I20260926 19:33:51.951481 14439 consensus_queue.cc:260] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.953166 14439 ts_tablet_manager.cc:1428] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260926 19:33:51.956321 14441 raft_consensus.cc:348] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.956796 14441 raft_consensus.cc:374] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:51.956897 14441 raft_consensus.cc:729] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f3f61459da425bb46983f15833d8a4, State: Initialized, Role: FOLLOWER
I20260926 19:33:51.956645 14440 raft_consensus.cc:348] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.957072 14440 raft_consensus.cc:374] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:51.957160 14440 raft_consensus.cc:729] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 361e1a07f3014ab2b12ce6ff07fc3164, State: Initialized, Role: FOLLOWER
I20260926 19:33:51.957867 14440 consensus_queue.cc:260] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.958602 14441 consensus_queue.cc:260] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.959993 14264 heartbeater.cc:503] Master 127.6.215.254:45377 was elected leader, sending a full tablet report...
I20260926 19:33:51.963598 14441 ts_tablet_manager.cc:1428] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Time spent starting tablet: real 0.020s user 0.007s sys 0.007s
I20260926 19:33:51.965304 14445 raft_consensus.cc:482] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:51.965581 14445 raft_consensus.cc:504] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.967150 14445 leader_election.cc:288] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579), 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:51.969831 14440 ts_tablet_manager.cc:1428] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Time spent starting tablet: real 0.026s user 0.012s sys 0.007s
I20260926 19:33:51.970204 14405 heartbeater.cc:503] Master 127.6.215.254:45377 was elected leader, sending a full tablet report...
I20260926 19:33:51.979872 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.967778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:51035 (local address 127.6.215.194:45579)
0926 19:33:51.968068 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.968076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.972503 (+ 4427us) server_negotiation.cc:407] Connection header received
0926 19:33:51.972884 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.972891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.973001 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.973076 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:51.973902 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.974982 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.977934 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.979036 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.979147 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.979197 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.979599 (+ 402us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.979686 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260926 19:33:51.980113 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.967414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:45579 (local address 127.6.215.193:51035)
0926 19:33:51.972399 (+ 4985us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.972420 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.972679 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.973278 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.973282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.973307 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:51.973692 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.973705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.975188 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.975194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.977749 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.977762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.979238 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.979246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.979433 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.979468 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.979584 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.979730 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4849,"mutex_wait_us":2,"thread_start_us":7134,"threads_started":1}
I20260926 19:33:51.981168 14219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82df00947c949c19d6017a8ac874b31" candidate_uuid: "f948ce0c2b6149528c17d39f278cfe76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f3f61459da425bb46983f15833d8a4" is_pre_election: true
I20260926 19:33:51.981680 14219 raft_consensus.cc:2454] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f948ce0c2b6149528c17d39f278cfe76 in term 0.
I20260926 19:33:51.982491 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.967827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:44321 (local address 127.6.215.195:36411)
0926 19:33:51.968066 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:51.968076 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:51.974979 (+ 6903us) server_negotiation.cc:407] Connection header received
0926 19:33:51.975164 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:51.975171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:51.975242 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:51.975321 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:51.977599 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.978646 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.980704 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.981597 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.981724 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.981771 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:51.982175 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 19:33:51.982310 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":772,"server-negotiator.queue_time_us":41,"spinlock_wait_cycles":9472}
I20260926 19:33:51.982425 14004 leader_election.cc:302] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f3f61459da425bb46983f15833d8a4, f948ce0c2b6149528c17d39f278cfe76; no voters:
I20260926 19:33:51.982690 14449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:51.967468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:36411 (local address 127.6.215.193:44321)
0926 19:33:51.974825 (+ 7357us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:51.974840 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:51.975034 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:51.975427 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:51.975435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:51.975458 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:51.977441 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.977451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.978782 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.978786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.980564 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:51.980575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:51.981807 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:51.981812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:51.981984 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:51.982019 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:33:51.982115 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 19:33:51.982249 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7232,"thread_start_us":4677,"threads_started":1}
I20260926 19:33:51.982909 14446 raft_consensus.cc:2781] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:51.983069 14446 raft_consensus.cc:482] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:51.983234 14446 raft_consensus.cc:3037] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:51.983589 14360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82df00947c949c19d6017a8ac874b31" candidate_uuid: "f948ce0c2b6149528c17d39f278cfe76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" is_pre_election: true
I20260926 19:33:51.984095 14360 raft_consensus.cc:2454] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f948ce0c2b6149528c17d39f278cfe76 in term 0.
I20260926 19:33:51.986099 14446 raft_consensus.cc:504] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.986675 14446 leader_election.cc:288] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [CANDIDATE]: Term 1 election: Requested vote from peers 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579), 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:51.987092 14219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82df00947c949c19d6017a8ac874b31" candidate_uuid: "f948ce0c2b6149528c17d39f278cfe76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f3f61459da425bb46983f15833d8a4"
I20260926 19:33:51.987092 14360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82df00947c949c19d6017a8ac874b31" candidate_uuid: "f948ce0c2b6149528c17d39f278cfe76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361e1a07f3014ab2b12ce6ff07fc3164"
I20260926 19:33:51.987288 14360 raft_consensus.cc:3037] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:51.987289 14219 raft_consensus.cc:3037] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:51.990928 14219 raft_consensus.cc:2454] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f948ce0c2b6149528c17d39f278cfe76 in term 1.
I20260926 19:33:51.990928 14360 raft_consensus.cc:2454] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f948ce0c2b6149528c17d39f278cfe76 in term 1.
I20260926 19:33:51.991307 14002 leader_election.cc:302] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 361e1a07f3014ab2b12ce6ff07fc3164, f948ce0c2b6149528c17d39f278cfe76; no voters:
I20260926 19:33:51.991658 14446 raft_consensus.cc:2781] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:51.991843 14446 raft_consensus.cc:686] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 1 LEADER]: Becoming Leader. State: Replica: f948ce0c2b6149528c17d39f278cfe76, State: Running, Role: LEADER
I20260926 19:33:51.992378 14446 consensus_queue.cc:237] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:51.997430 13149 catalog_manager.cc:5495] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 reported cstate change: term changed from 0 to 1, leader changed from <none> to f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193). New cstate: current_term: 1 leader_uuid: "f948ce0c2b6149528c17d39f278cfe76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } health_report { overall_health: UNKNOWN } } }
I20260926 19:33:52.030020 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:52.024391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.6.215.193:34191)
0926 19:33:52.024568 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:52.024573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:52.024634 (+ 61us) server_negotiation.cc:407] Connection header received
0926 19:33:52.024858 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:52.024868 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:52.024953 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:52.025083 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:52.025417 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.026091 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.027320 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.027755 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.027884 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:52.028486 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:52.029318 (+ 832us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:52.029482 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:52.029625 (+ 143us) server_negotiation.cc:299] Negotiation successful
0926 19:33:52.029712 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 19:33:52.038177 14360 raft_consensus.cc:1261] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Refusing update from remote peer f948ce0c2b6149528c17d39f278cfe76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:52.038190 14219 raft_consensus.cc:1261] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 1 FOLLOWER]: Refusing update from remote peer f948ce0c2b6149528c17d39f278cfe76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:52.039090 14446 consensus_queue.cc:1035] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:52.039355 14445 consensus_queue.cc:1035] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:33:52.050705 14408 tablet.cc:2369] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:52.052901 14454 mvcc.cc:205] Tried to move back new op lower bound from 7333688246419238912 to 7333688246245208064. Current Snapshot: MvccSnapshot[applied={T|T < 7333688246419238912}]
I20260926 19:33:52.057194 14456 mvcc.cc:205] Tried to move back new op lower bound from 7333688246419238912 to 7333688246245208064. Current Snapshot: MvccSnapshot[applied={T|T < 7333688246419238912}]
I20260926 19:33:52.071419 13150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:52.071702 13150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:52.077809 13150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd34298261224bda8de2cb4ffa00ce2f: Sending DeleteTablet for 3 replicas of tablet a82df00947c949c19d6017a8ac874b31
I20260926 19:33:52.079015 14051 tablet_service.cc:1514] Processing DeleteTablet for tablet a82df00947c949c19d6017a8ac874b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:52 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43166
I20260926 19:33:52.079108 14340 tablet_service.cc:1514] Processing DeleteTablet for tablet a82df00947c949c19d6017a8ac874b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:52 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:46176
I20260926 19:33:52.079670 14199 tablet_service.cc:1514] Processing DeleteTablet for tablet a82df00947c949c19d6017a8ac874b31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:52 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:49474
I20260926 19:33:52.090420 14465 tablet_replica.cc:330] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: stopping tablet replica
I20260926 19:33:52.090900 14465 raft_consensus.cc:2229] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:52.091379 14464 tablet_replica.cc:330] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: stopping tablet replica
I20260926 19:33:52.091424 14465 raft_consensus.cc:2258] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:52.091677 14463 tablet_replica.cc:330] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: stopping tablet replica
I20260926 19:33:52.092819 14463 raft_consensus.cc:2229] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:52.092257 14464 raft_consensus.cc:2229] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:52.093587 14464 raft_consensus.cc:2258] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:52.093889 14465 ts_tablet_manager.cc:1905] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:52.094429 14463 raft_consensus.cc:2258] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:52.096943 14464 ts_tablet_manager.cc:1905] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:52.098874 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 13118
I20260926 19:33:52.099349 14463 ts_tablet_manager.cc:1905] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:52.100056 13180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:52.102152 14465 ts_tablet_manager.cc:1918] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:52.102380 14465 log.cc:1196] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/a82df00947c949c19d6017a8ac874b31
I20260926 19:33:52.102989 14465 ts_tablet_manager.cc:1939] T a82df00947c949c19d6017a8ac874b31 P 07f3f61459da425bb46983f15833d8a4: Deleting consensus metadata
I20260926 19:33:52.111929 14464 ts_tablet_manager.cc:1918] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:52.112112 14464 log.cc:1196] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/a82df00947c949c19d6017a8ac874b31
I20260926 19:33:52.112671 14464 ts_tablet_manager.cc:1939] T a82df00947c949c19d6017a8ac874b31 P 361e1a07f3014ab2b12ce6ff07fc3164: Deleting consensus metadata
I20260926 19:33:52.117064 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:52.082575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.6.215.253:33967)
0926 19:33:52.082795 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:52.082804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:52.097498 (+ 14694us) server_negotiation.cc:407] Connection header received
0926 19:33:52.098150 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:52.098157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:52.098259 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:52.098339 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:52.100753 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.101905 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.104070 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.104712 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.104855 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:52.108850 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:52.108857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:52.108862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:52.108903 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:52.112940 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.114026 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.114032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.114036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.114175 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.114686 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.114692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.114695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.115034 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:52.115251 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:52.116453 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0926 19:33:52.116671 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:33:52.120838 14463 ts_tablet_manager.cc:1918] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:52.121052 14463 log.cc:1196] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/a82df00947c949c19d6017a8ac874b31
I20260926 19:33:52.122083 14463 ts_tablet_manager.cc:1939] T a82df00947c949c19d6017a8ac874b31 P f948ce0c2b6149528c17d39f278cfe76: Deleting consensus metadata
I20260926 19:33:52.133611 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:52.097653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.6.215.252:34391)
0926 19:33:52.097934 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:52.097940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:52.118123 (+ 20183us) server_negotiation.cc:407] Connection header received
0926 19:33:52.118367 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:52.118373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:52.118476 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:52.118547 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:52.120486 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.122037 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.123930 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.124539 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.124636 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:52.127562 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:52.127569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:52.127578 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:52.127611 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:52.130282 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.130942 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.130947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.130951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.131093 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.131507 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.131515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.131518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.131790 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:52.132084 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:52.132429 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 19:33:52.132691 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:33:52.274622 13136 catalog_manager.cc:4841] TS 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579): tablet a82df00947c949c19d6017a8ac874b31 (table test-table [id=5607dda2061249aa9b38e5124aeea823]) successfully deleted
I20260926 19:33:52.275926 13134 catalog_manager.cc:4841] TS 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411): tablet a82df00947c949c19d6017a8ac874b31 (table test-table [id=5607dda2061249aa9b38e5124aeea823]) successfully deleted
I20260926 19:33:52.275954 13135 catalog_manager.cc:4841] TS f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191): tablet a82df00947c949c19d6017a8ac874b31 (table test-table [id=5607dda2061249aa9b38e5124aeea823]) successfully deleted
I20260926 19:33:52.276612 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 13118
I20260926 19:33:52.319893 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:52.303330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.6.215.252:34391)
0926 19:33:52.303572 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:52.303579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:52.304426 (+ 847us) server_negotiation.cc:407] Connection header received
0926 19:33:52.304625 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:52.304630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:52.304703 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:52.304848 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:52.306427 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.307160 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.309026 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.309684 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.309808 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:52.312802 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:52.312809 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:52.312814 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:52.312851 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:52.316630 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.317465 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.317470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.317473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.317622 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.318345 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.318351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.318355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.318734 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:52.319024 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:52.319341 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 19:33:52.319545 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:33:52.323330 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:52.308070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.6.215.253:33967)
0926 19:33:52.308255 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:52.308261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:52.308434 (+ 173us) server_negotiation.cc:407] Connection header received
0926 19:33:52.308604 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:52.308610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:52.308692 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:52.308821 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:52.310776 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.311869 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.313674 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:52.314231 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:52.314386 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:52.317050 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:52.317060 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:52.317064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:52.317095 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:52.320625 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.321407 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.321414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.321417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.321554 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:52.321900 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:52.321904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:52.321908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:52.322253 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:52.322499 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:52.322754 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 19:33:52.322968 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 19:33:53.057662 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.056981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.193:34155)
0926 19:33:53.057386 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.057502 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":204,"threads_started":1}
W20260926 19:33:53.058158 14116 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:45377 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.066707 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.065785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.195:35749)
0926 19:33:53.066376 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.066523 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":409,"threads_started":1}
W20260926 19:33:53.067092 14405 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:45377 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.067133 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.066027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:59593)
0926 19:33:53.066805 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.066990 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":558,"threads_started":1}
W20260926 19:33:53.067548 14264 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:45377 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.578763 14477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd34298261224bda8de2cb4ffa00ce2f)
I20260926 19:33:53.578922 14478 raft_consensus.cc:482] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd34298261224bda8de2cb4ffa00ce2f)
I20260926 19:33:53.579109 14477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } }
I20260926 19:33:53.579232 14478 raft_consensus.cc:504] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } }
I20260926 19:33:53.580516 14478 leader_election.cc:288] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd34298261224bda8de2cb4ffa00ce2f (127.6.215.254:45377), 3f9ef2395678483bb14fe6b9aa5ec209 (127.6.215.252:34391)
I20260926 19:33:53.580581 14477 leader_election.cc:288] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd34298261224bda8de2cb4ffa00ce2f (127.6.215.254:45377), d44b3986271c40009d8c47e02ff35329 (127.6.215.253:33967)
I20260926 19:33:53.581878 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.580690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55598)
0926 19:33:53.581287 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.581553 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":250,"threads_started":1}
I20260926 19:33:53.592470 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.581300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33967 (local address 127.0.0.1:49066)
0926 19:33:53.582125 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.582147 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:53.582305 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:53.583340 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:53.583348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:53.583387 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:53.583710 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.583725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.585997 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.586006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.588851 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.588868 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.591297 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.591308 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.591610 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.591672 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 19:33:53.591756 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 19:33:53.591962 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":5597,"threads_started":1}
I20260926 19:33:53.592538 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.580808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55590)
0926 19:33:53.591944 (+ 11136us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.592176 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10954,"threads_started":1}
W20260926 19:33:53.593502 13922 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.593653 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.581584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49066 (local address 127.6.215.253:33967)
0926 19:33:53.582909 (+ 1325us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:53.582918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:53.582943 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:33:53.583029 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:53.583037 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:53.583133 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:53.583874 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:53.584167 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.585791 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.589146 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.590969 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.592846 (+ 1877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.593000 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:53.593071 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 19:33:53.593240 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1166,"thread_start_us":987,"threads_started":1}
W20260926 19:33:53.594295 13735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.594894 13760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d44b3986271c40009d8c47e02ff35329" is_pre_election: true
I20260926 19:33:53.595279 13760 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f9ef2395678483bb14fe6b9aa5ec209 in term 1.
I20260926 19:33:53.595592 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.595208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55608)
0926 19:33:53.595346 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.595429 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260926 19:33:53.596022 13922 leader_election.cc:334] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd34298261224bda8de2cb4ffa00ce2f (127.6.215.254:45377): Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.596417 13922 leader_election.cc:302] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f9ef2395678483bb14fe6b9aa5ec209, d44b3986271c40009d8c47e02ff35329; no voters: cd34298261224bda8de2cb4ffa00ce2f
I20260926 19:33:53.596545 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.595960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55622)
0926 19:33:53.596144 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.596333 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 19:33:53.596722 13735 leader_election.cc:334] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd34298261224bda8de2cb4ffa00ce2f (127.6.215.254:45377): Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.596823 14477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 19:33:53.596947 14477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd34298261224bda8de2cb4ffa00ce2f)
I20260926 19:33:53.597046 14477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:33:53.599905 14477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } }
I20260926 19:33:53.600724 14477 leader_election.cc:288] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [CANDIDATE]: Term 2 election: Requested vote from peers cd34298261224bda8de2cb4ffa00ce2f (127.6.215.254:45377), d44b3986271c40009d8c47e02ff35329 (127.6.215.253:33967)
I20260926 19:33:53.601052 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.581250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:34391 (local address 127.0.0.1:43448)
0926 19:33:53.593339 (+ 12089us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.593360 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:53.593509 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:53.594507 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:53.594512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:53.594538 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:53.594895 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.594918 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.597041 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.597045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.598381 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.598400 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.600129 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.600157 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.600314 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.600355 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 19:33:53.600551 (+ 196us) client_negotiation.cc:240] Negotiation successful
0926 19:33:53.600707 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12001,"thread_start_us":11594,"threads_started":1}
I20260926 19:33:53.601315 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.587122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.6.215.252:34391)
0926 19:33:53.593524 (+ 6402us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:53.593530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:53.593550 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:33:53.593624 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:53.593631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:53.593714 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:53.595353 (+ 1639us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:53.595622 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.596835 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.598577 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.599969 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.600099 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.600213 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:53.600591 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 19:33:53.600746 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6264,"thread_start_us":6114,"threads_started":1}
I20260926 19:33:53.601361 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.600929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55630)
0926 19:33:53.601117 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.601205 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 19:33:53.601712 13760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d44b3986271c40009d8c47e02ff35329"
I20260926 19:33:53.601936 13760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:33:53.602101 13947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d44b3986271c40009d8c47e02ff35329" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" is_pre_election: true
I20260926 19:33:53.602468 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.602101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55636)
0926 19:33:53.602246 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.602316 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260926 19:33:53.602491 13947 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d44b3986271c40009d8c47e02ff35329 in current term 2: Already voted for candidate 3f9ef2395678483bb14fe6b9aa5ec209 in this term.
W20260926 19:33:53.602952 13922 leader_election.cc:334] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cd34298261224bda8de2cb4ffa00ce2f (127.6.215.254:45377): Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
I20260926 19:33:53.603478 13736 leader_election.cc:302] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d44b3986271c40009d8c47e02ff35329; no voters: 3f9ef2395678483bb14fe6b9aa5ec209, cd34298261224bda8de2cb4ffa00ce2f
I20260926 19:33:53.604869 13760 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f9ef2395678483bb14fe6b9aa5ec209 in term 2.
I20260926 19:33:53.605050 14478 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 19:33:53.605281 13922 leader_election.cc:302] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f9ef2395678483bb14fe6b9aa5ec209, d44b3986271c40009d8c47e02ff35329; no voters: cd34298261224bda8de2cb4ffa00ce2f
I20260926 19:33:53.605540 14477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:33:53.606015 14477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [term 2 LEADER]: Becoming Leader. State: Replica: 3f9ef2395678483bb14fe6b9aa5ec209, State: Running, Role: LEADER
I20260926 19:33:53.606421 14477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } }
I20260926 19:33:53.608335 14487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f9ef2395678483bb14fe6b9aa5ec209. Latest consensus state: current_term: 2 leader_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:53.608731 14487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:53.609429 14489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:53.613039 14489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:53.613852 14489 catalog_manager.cc:1259] Loaded cluster ID: c865ba0a47534565a1cbb132a5047d56
I20260926 19:33:53.613938 14489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:53.615025 14489 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:53.615739 14489 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Loaded TSK: 0
I20260926 19:33:53.616632 14489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:33:53.670186 13937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:53.677021 13937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:53.681041 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.680612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55674)
0926 19:33:53.680838 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.680916 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 19:33:53.681036 13760 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [term 2 FOLLOWER]: Refusing update from remote peer 3f9ef2395678483bb14fe6b9aa5ec209: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 19:33:53.682018 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.681627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55680)
0926 19:33:53.681784 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.681867 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 19:33:53.682184 14487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 19:33:53.682449 13922 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 -> Peer cd34298261224bda8de2cb4ffa00ce2f (127.6.215.254:45377): Couldn't send request to peer cd34298261224bda8de2cb4ffa00ce2f. Status: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 19:33:53.685022 14478 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f9ef2395678483bb14fe6b9aa5ec209. Latest consensus state: current_term: 2 leader_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:53.685307 14478 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:33:53.689299 14477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:53.689589 14477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209 [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:53.690182 14478 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd34298261224bda8de2cb4ffa00ce2f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45377 } } peers { permanent_uuid: "d44b3986271c40009d8c47e02ff35329" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33967 } attrs { promote: false } } peers { permanent_uuid: "3f9ef2395678483bb14fe6b9aa5ec209" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 34391 } attrs { promote: false } } }
I20260926 19:33:53.690445 14478 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44b3986271c40009d8c47e02ff35329 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:33:53.709211 14497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.698644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:36411 (local address 127.0.0.1:46182)
0926 19:33:53.700133 (+ 1489us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.700155 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:53.700402 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:53.702271 (+ 1869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:53.702277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:53.702309 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:53.702733 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.702742 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.704017 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.704025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.706383 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.706398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.708230 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.708243 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.708432 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.708473 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 19:33:53.708547 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 19:33:53.708696 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"mutex_wait_us":1,"thread_start_us":1153,"threads_started":1}
I20260926 19:33:53.710175 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.699036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46182 (local address 127.6.215.195:36411)
0926 19:33:53.701841 (+ 2805us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:53.701848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:53.701869 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:33:53.701937 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:53.701941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:53.702051 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:53.702253 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:53.702903 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.703857 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.706580 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.707961 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.709514 (+ 1553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.709630 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:53.709696 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 19:33:53.709852 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2657,"thread_start_us":2519,"threads_started":1}
I20260926 19:33:53.711160 14340 tablet_service.cc:1467] Processing CreateTablet for tablet 33211766d0f84d2396feefa1460e026f (DEFAULT_TABLE table=test-table [id=a3960bb549814350b4f28b87e232cd3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:53.711407 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.698257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34191 (local address 127.0.0.1:43192)
0926 19:33:53.698785 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.698805 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:53.698909 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:53.701859 (+ 2950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:53.701867 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:53.701902 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:53.702448 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.702463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.705146 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.705152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.707412 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.707427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.710645 (+ 3218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.710651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.710835 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.710858 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 19:33:53.710966 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 19:33:53.711098 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":30}
I20260926 19:33:53.711712 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.699142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.6.215.193:34191)
0926 19:33:53.701420 (+ 2278us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:53.701429 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:53.701453 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:33:53.701536 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:53.701544 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:53.701680 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:53.702711 (+ 1031us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:53.702913 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.704968 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.707648 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.710124 (+ 2476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.710614 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.710723 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:53.710994 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 19:33:53.711123 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2063,"thread_start_us":1964,"threads_started":1}
I20260926 19:33:53.711751 14340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33211766d0f84d2396feefa1460e026f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:53.712358 14051 tablet_service.cc:1467] Processing CreateTablet for tablet 33211766d0f84d2396feefa1460e026f (DEFAULT_TABLE table=test-table [id=a3960bb549814350b4f28b87e232cd3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:53.712745 14051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33211766d0f84d2396feefa1460e026f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:53.719218 14501 tablet_bootstrap.cc:492] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Bootstrap starting.
I20260926 19:33:53.720285 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.698009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:45579 (local address 127.0.0.1:49486)
0926 19:33:53.698287 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.698317 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:53.698474 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:53.713595 (+ 15121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:53.713601 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:53.713627 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:53.713805 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.713816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.715322 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.715327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.717219 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.717233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.719289 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.719299 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.719556 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.719614 (+ 58us) client_negotiation.cc:769] Sending connection context
0926 19:33:53.719697 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 19:33:53.719888 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260926 19:33:53.720822 14501 tablet_bootstrap.cc:654] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:53.721325 14500 tablet_bootstrap.cc:492] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: Bootstrap starting.
I20260926 19:33:53.721397 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.698067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49486 (local address 127.6.215.194:45579)
0926 19:33:53.713230 (+ 15163us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:53.713239 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:53.713261 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:33:53.713346 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:53.713354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:53.713436 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:53.713939 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:53.714078 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.715146 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.717405 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.719019 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.720711 (+ 1692us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.720871 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:53.720956 (+ 85us) server_negotiation.cc:299] Negotiation successful
0926 19:33:53.721097 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15037,"thread_start_us":14935,"threads_started":1}
I20260926 19:33:53.722143 14199 tablet_service.cc:1467] Processing CreateTablet for tablet 33211766d0f84d2396feefa1460e026f (DEFAULT_TABLE table=test-table [id=a3960bb549814350b4f28b87e232cd3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:53.722532 14199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33211766d0f84d2396feefa1460e026f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:53.723054 14501 tablet_bootstrap.cc:492] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: No bootstrap required, opened a new log
I20260926 19:33:53.723147 14500 tablet_bootstrap.cc:654] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:53.723201 14501 ts_tablet_manager.cc:1397] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260926 19:33:53.724017 14501 raft_consensus.cc:348] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.724251 14501 raft_consensus.cc:374] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:53.724373 14501 raft_consensus.cc:729] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f948ce0c2b6149528c17d39f278cfe76, State: Initialized, Role: FOLLOWER
I20260926 19:33:53.724611 14501 consensus_queue.cc:260] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.725489 14500 tablet_bootstrap.cc:492] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: No bootstrap required, opened a new log
I20260926 19:33:53.725659 14500 ts_tablet_manager.cc:1397] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 19:33:53.726661 14500 raft_consensus.cc:348] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.726979 14500 raft_consensus.cc:374] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:53.727095 14500 raft_consensus.cc:729] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 361e1a07f3014ab2b12ce6ff07fc3164, State: Initialized, Role: FOLLOWER
I20260926 19:33:53.727378 14500 consensus_queue.cc:260] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.728405 14500 ts_tablet_manager.cc:1428] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260926 19:33:53.733319 14406 heartbeater.cc:503] Master 127.6.215.252:34391 was elected leader, sending a full tablet report...
I20260926 19:33:53.734779 14505 tablet_bootstrap.cc:492] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Bootstrap starting.
I20260926 19:33:53.736649 14505 tablet_bootstrap.cc:654] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:53.738581 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.728639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.195:60871)
0926 19:33:53.738195 (+ 9556us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.738344 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9430,"thread_start_us":9334,"threads_started":1}
I20260926 19:33:53.739235 14501 ts_tablet_manager.cc:1428] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Time spent starting tablet: real 0.016s user 0.004s sys 0.012s
I20260926 19:33:53.740530 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.740139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.195:54739)
0926 19:33:53.740310 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.740402 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 19:33:53.740358 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.739533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.193:55003)
0926 19:33:53.739996 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.740111 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":255,"threads_started":1}
I20260926 19:33:53.741926 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.741538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.193:44525)
0926 19:33:53.741729 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.741801 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 19:33:53.742483 14117 heartbeater.cc:503] Master 127.6.215.252:34391 was elected leader, sending a full tablet report...
I20260926 19:33:53.743342 14505 tablet_bootstrap.cc:492] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: No bootstrap required, opened a new log
I20260926 19:33:53.743499 14505 ts_tablet_manager.cc:1397] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260926 19:33:53.744401 14505 raft_consensus.cc:348] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.744604 14505 raft_consensus.cc:374] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:53.744711 14505 raft_consensus.cc:729] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f3f61459da425bb46983f15833d8a4, State: Initialized, Role: FOLLOWER
I20260926 19:33:53.745007 14505 consensus_queue.cc:260] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.745893 14505 ts_tablet_manager.cc:1428] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 19:33:53.747277 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.746318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:49101)
0926 19:33:53.746849 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.747026 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":279,"threads_started":1}
I20260926 19:33:53.747956 14265 heartbeater.cc:503] Master 127.6.215.252:34391 was elected leader, sending a full tablet report...
I20260926 19:33:53.748225 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.747851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:36055)
0926 19:33:53.748017 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.748094 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 19:33:53.824952 14119 tablet.cc:2369] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:53.935477 14510 raft_consensus.cc:482] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:53.935761 14510 raft_consensus.cc:504] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.937486 14510 leader_election.cc:288] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191), 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:53.944177 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.937687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34191 (local address 127.6.215.194:55941)
0926 19:33:53.937985 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.938008 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:53.938294 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:53.938765 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:53.938770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:53.938804 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:53.939179 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.939190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.940280 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.940286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.942236 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.942247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.943439 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.943445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.943638 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.943681 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 19:33:53.943780 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 19:33:53.943900 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":3}
I20260926 19:33:53.944274 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.937817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:55941 (local address 127.6.215.193:34191)
0926 19:33:53.938010 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:53.938016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:53.938115 (+ 99us) server_negotiation.cc:407] Connection header received
0926 19:33:53.938519 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:53.938527 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:53.938618 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:53.938718 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:53.939324 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.940134 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.942375 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.943220 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.943345 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.943398 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:53.943885 (+ 487us) server_negotiation.cc:299] Negotiation successful
0926 19:33:53.944023 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 19:33:53.944481 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.938135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:38927 (local address 127.6.215.195:36411)
0926 19:33:53.938343 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:53.938351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:53.938624 (+ 273us) server_negotiation.cc:407] Connection header received
0926 19:33:53.938791 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:53.938797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:53.938872 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:53.938987 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:53.939575 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.940312 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.942529 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.943493 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.943671 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.943742 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:53.944093 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 19:33:53.944241 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 19:33:53.944624 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.937830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:36411 (local address 127.6.215.194:38927)
0926 19:33:53.938510 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.938527 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:53.938649 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:53.939010 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:53.939013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:53.939032 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:53.939419 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.939433 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.940454 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.940457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.942362 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:53.942375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:53.943716 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:53.943721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:53.943926 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:53.943959 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 19:33:53.944051 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 19:33:53.944176 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":24,"thread_start_us":358,"threads_started":1}
I20260926 19:33:53.945047 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33211766d0f84d2396feefa1460e026f" candidate_uuid: "07f3f61459da425bb46983f15833d8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f948ce0c2b6149528c17d39f278cfe76" is_pre_election: true
I20260926 19:33:53.945255 14360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33211766d0f84d2396feefa1460e026f" candidate_uuid: "07f3f61459da425bb46983f15833d8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" is_pre_election: true
I20260926 19:33:53.945473 14071 raft_consensus.cc:2454] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f3f61459da425bb46983f15833d8a4 in term 0.
I20260926 19:33:53.945515 14360 raft_consensus.cc:2454] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07f3f61459da425bb46983f15833d8a4 in term 0.
I20260926 19:33:53.946147 14152 leader_election.cc:302] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 07f3f61459da425bb46983f15833d8a4, 361e1a07f3014ab2b12ce6ff07fc3164, f948ce0c2b6149528c17d39f278cfe76; no voters:
I20260926 19:33:53.946556 14510 raft_consensus.cc:2781] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:53.946694 14510 raft_consensus.cc:482] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:53.946789 14510 raft_consensus.cc:3037] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:53.948977 14510 raft_consensus.cc:504] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.949649 14510 leader_election.cc:288] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [CANDIDATE]: Term 1 election: Requested vote from peers f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191), 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411)
I20260926 19:33:53.950213 14071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33211766d0f84d2396feefa1460e026f" candidate_uuid: "07f3f61459da425bb46983f15833d8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f948ce0c2b6149528c17d39f278cfe76"
I20260926 19:33:53.950292 14360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33211766d0f84d2396feefa1460e026f" candidate_uuid: "07f3f61459da425bb46983f15833d8a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "361e1a07f3014ab2b12ce6ff07fc3164"
I20260926 19:33:53.950435 14071 raft_consensus.cc:3037] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:53.950492 14360 raft_consensus.cc:3037] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:53.953476 14360 raft_consensus.cc:2454] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f3f61459da425bb46983f15833d8a4 in term 1.
I20260926 19:33:53.953476 14071 raft_consensus.cc:2454] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f3f61459da425bb46983f15833d8a4 in term 1.
I20260926 19:33:53.953917 14153 leader_election.cc:302] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f3f61459da425bb46983f15833d8a4, f948ce0c2b6149528c17d39f278cfe76; no voters:
I20260926 19:33:53.954259 14510 raft_consensus.cc:2781] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:53.954731 14510 raft_consensus.cc:686] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 1 LEADER]: Becoming Leader. State: Replica: 07f3f61459da425bb46983f15833d8a4, State: Running, Role: LEADER
I20260926 19:33:53.955158 14510 consensus_queue.cc:237] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } }
I20260926 19:33:53.955731 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:53.955327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:57411)
0926 19:33:53.955497 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:53.955601 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 19:33:53.958663 13937 catalog_manager.cc:5495] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07f3f61459da425bb46983f15833d8a4 (127.6.215.194). New cstate: current_term: 1 leader_uuid: "07f3f61459da425bb46983f15833d8a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f3f61459da425bb46983f15833d8a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 45579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 } health_report { overall_health: UNKNOWN } } }
W20260926 19:33:53.965317 14267 tablet.cc:2369] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:54.010799 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.005855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.6.215.194:45579)
0926 19:33:54.006094 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:54.006099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:54.006113 (+ 14us) server_negotiation.cc:407] Connection header received
0926 19:33:54.006194 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:54.006198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:54.006275 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:54.006446 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:54.006782 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.007479 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.008729 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.009186 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.009314 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:54.009714 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:54.010237 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:54.010395 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:54.010455 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 19:33:54.010533 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:33:54.015287 14360 raft_consensus.cc:1261] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Refusing update from remote peer 07f3f61459da425bb46983f15833d8a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:54.015552 14071 raft_consensus.cc:1261] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 1 FOLLOWER]: Refusing update from remote peer 07f3f61459da425bb46983f15833d8a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:54.016080 14513 consensus_queue.cc:1035] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361e1a07f3014ab2b12ce6ff07fc3164" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 36411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:54.016448 14510 consensus_queue.cc:1035] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f948ce0c2b6149528c17d39f278cfe76" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:54.019263 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.018743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.193:45769)
0926 19:33:54.018960 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.019070 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260926 19:33:54.019263 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.018743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.195:42789)
0926 19:33:54.018967 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.019070 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260926 19:33:54.027019 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.026242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:42977)
0926 19:33:54.026666 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.026798 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280}
I20260926 19:33:54.027837 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.027407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.195:44385)
0926 19:33:54.027609 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.027685 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260926 19:33:54.037135 13936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:54.037441 13936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:54.045174 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.044532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.194:47523)
0926 19:33:54.044867 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.044994 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260926 19:33:54.045693 13936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f9ef2395678483bb14fe6b9aa5ec209: Sending DeleteTablet for 3 replicas of tablet 33211766d0f84d2396feefa1460e026f
I20260926 19:33:54.046959 14051 tablet_service.cc:1514] Processing DeleteTablet for tablet 33211766d0f84d2396feefa1460e026f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:54 UTC) from {username='oryx', principal='oryx/127.6.215.252@KRBTEST.COM'} at 127.0.0.1:43192
I20260926 19:33:54.047405 14199 tablet_service.cc:1514] Processing DeleteTablet for tablet 33211766d0f84d2396feefa1460e026f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:54 UTC) from {username='oryx', principal='oryx/127.6.215.252@KRBTEST.COM'} at 127.0.0.1:49486
I20260926 19:33:54.047976 14340 tablet_service.cc:1514] Processing DeleteTablet for tablet 33211766d0f84d2396feefa1460e026f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:54 UTC) from {username='oryx', principal='oryx/127.6.215.252@KRBTEST.COM'} at 127.0.0.1:46182
I20260926 19:33:54.048911 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 13909
I20260926 19:33:54.055081 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.054576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.6.215.193:49015)
0926 19:33:54.054812 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.054910 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260926 19:33:54.068231 14525 tablet_replica.cc:330] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: stopping tablet replica
I20260926 19:33:54.069296 14525 raft_consensus.cc:2229] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:54.069713 14527 tablet_replica.cc:330] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: stopping tablet replica
I20260926 19:33:54.069921 14527 raft_consensus.cc:2229] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:54.070256 14527 raft_consensus.cc:2258] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:54.067129 14526 tablet_replica.cc:330] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: stopping tablet replica
I20260926 19:33:54.070695 14526 raft_consensus.cc:2229] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:54.071470 14526 raft_consensus.cc:2258] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:54.071718 14525 raft_consensus.cc:2258] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:54.072717 14527 ts_tablet_manager.cc:1905] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:54.074074 14525 ts_tablet_manager.cc:1905] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:54.076267 14526 ts_tablet_manager.cc:1905] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:54.078856 14527 ts_tablet_manager.cc:1918] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:54.079013 14527 log.cc:1196] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/33211766d0f84d2396feefa1460e026f
I20260926 19:33:54.079478 14527 ts_tablet_manager.cc:1939] T 33211766d0f84d2396feefa1460e026f P 361e1a07f3014ab2b12ce6ff07fc3164: Deleting consensus metadata
I20260926 19:33:54.080703 13921 catalog_manager.cc:4841] TS 361e1a07f3014ab2b12ce6ff07fc3164 (127.6.215.195:36411): tablet 33211766d0f84d2396feefa1460e026f (table test-table [id=a3960bb549814350b4f28b87e232cd3b]) successfully deleted
I20260926 19:33:54.080973 14525 ts_tablet_manager.cc:1918] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:54.081101 14525 log.cc:1196] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/33211766d0f84d2396feefa1460e026f
I20260926 19:33:54.081518 14525 ts_tablet_manager.cc:1939] T 33211766d0f84d2396feefa1460e026f P f948ce0c2b6149528c17d39f278cfe76: Deleting consensus metadata
I20260926 19:33:54.082628 13922 catalog_manager.cc:4841] TS f948ce0c2b6149528c17d39f278cfe76 (127.6.215.193:34191): tablet 33211766d0f84d2396feefa1460e026f (table test-table [id=a3960bb549814350b4f28b87e232cd3b]) successfully deleted
I20260926 19:33:54.082891 14526 ts_tablet_manager.cc:1918] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:54.083036 14526 log.cc:1196] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/33211766d0f84d2396feefa1460e026f
I20260926 19:33:54.083096 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.050084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.6.215.193:34191)
0926 19:33:54.050287 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:54.050296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:54.053657 (+ 3361us) server_negotiation.cc:407] Connection header received
0926 19:33:54.053877 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:54.053886 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:54.053982 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:54.054090 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:54.067294 (+ 13204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.068671 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.070442 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.071117 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.071264 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:54.075026 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:54.075051 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:54.075075 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:54.075137 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:54.079751 (+ 4614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:54.080692 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:54.080702 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:54.080705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:54.080966 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:54.081414 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:54.081420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:54.081431 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:54.081866 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:54.082194 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:54.082485 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 19:33:54.082711 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45,"spinlock_wait_cycles":14592}
I20260926 19:33:54.083503 14526 ts_tablet_manager.cc:1939] T 33211766d0f84d2396feefa1460e026f P 07f3f61459da425bb46983f15833d8a4: Deleting consensus metadata
I20260926 19:33:54.083750 14111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:54.084726 13923 catalog_manager.cc:4841] TS 07f3f61459da425bb46983f15833d8a4 (127.6.215.194:45579): tablet 33211766d0f84d2396feefa1460e026f (table test-table [id=a3960bb549814350b4f28b87e232cd3b]) successfully deleted
I20260926 19:33:54.091147 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.090677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55692)
0926 19:33:54.090852 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.090954 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 19:33:54.092103 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.091748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55696)
0926 19:33:54.091932 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.092002 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 19:33:54.238181 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 13909
I20260926 19:33:54.260334 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 14128
I20260926 19:33:54.276405 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.261222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49498 (local address 127.6.215.194:45579)
0926 19:33:54.261409 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:54.261416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:54.261514 (+ 98us) server_negotiation.cc:407] Connection header received
0926 19:33:54.261661 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:54.261672 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:54.261770 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:54.261894 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:54.263225 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.264418 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.265949 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.266588 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.266721 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:54.269461 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:54.269479 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:54.269499 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:54.269541 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:54.273624 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:54.274307 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:54.274316 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:54.274320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:54.274499 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:54.274846 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:54.274851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:54.274854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:54.275321 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:54.275619 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:54.275793 (+ 174us) server_negotiation.cc:299] Negotiation successful
0926 19:33:54.276023 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 19:33:54.277153 14259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:54.403347 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 14128
I20260926 19:33:54.423419 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 14277
I20260926 19:33:54.437222 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.424202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.6.215.195:36411)
0926 19:33:54.424386 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:54.424392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:54.424413 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:33:54.424509 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:54.424519 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:54.424598 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:54.424708 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:54.426069 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.426781 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.428003 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:54.428388 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:54.428521 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:54.431517 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:54.431532 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:54.431547 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:54.431585 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:54.434712 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:54.435343 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:54.435351 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:54.435353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:54.435479 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:54.435785 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:54.435788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:54.435790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:54.436129 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:54.436333 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:54.436606 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 19:33:54.436762 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 19:33:54.437841 14400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:54.531431 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 14277
I20260926 19:33:54.532423 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.531844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55708)
0926 19:33:54.532132 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.532236 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260926 19:33:54.533931 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:54.533479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45377 (local address 127.0.0.1:55712)
0926 19:33:54.533657 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:54.533771 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 19:33:54.549968 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 13612
I20260926 19:33:54.551029 13780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:54.665372 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 13612
I20260926 19:33:54.685693 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 13713
I20260926 19:33:54.686800 13967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:33:54.805128 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 13713
2026-09-26T19:33:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9683 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:33:54 dist-test-slave-phbh krb5kdc[14537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:33:54 dist-test-slave-phbh krb5kdc[14537](info): set up 2 sockets
Sep 26 19:33:54 dist-test-slave-phbh krb5kdc[14537](info): commencing operation
krb5kdc: starting...
W20260926 19:33:55.909704 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:33:56 dist-test-slave-phbh krb5kdc[14537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:33:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:33:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab.
Entry for principal oryx/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab.
I20260926 19:33:56.131757 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:39141
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab
--principal=oryx/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:39141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:56.451596 14550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:56.451949 14550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:56.452118 14550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:56.459312 14550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:33:56.459471 14550 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 19:33:56.459570 14550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:56.459688 14550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:56.459797 14550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:33:56.459895 14550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:33:56.470021 14550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:39141
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.254.keytab
--principal=oryx/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:39141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:56.471465 14550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:56.472934 14550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:56.485116 14555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:56.485080 14558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:56.485037 14556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:56.486037 14550 server_base.cc:1034] running on GCE node
I20260926 19:33:56.486899 14550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:56.488286 14550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:56.489526 14550 hybrid_clock.cc:648] HybridClock initialized: now 1790451236489470 us; error 56 us; skew 500 ppm
I20260926 19:33:56.489859 14550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:56 dist-test-slave-phbh krb5kdc[14537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451236, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:56.492926 14550 init.cc:377] Logged in from keytab as oryx/127.6.215.254@KRBTEST.COM (short username oryx)
I20260926 19:33:56.494392 14550 webserver.cc:460] Webserver started at http://127.6.215.254:36027/ using document root <none> and password file <none>
I20260926 19:33:56.495093 14550 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:56.495222 14550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:56.495553 14550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:56.498513 14550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "7c1447da7f0040fda9bab90ce1a972fa"
format_stamp: "Formatted at 2026-09-26 19:33:56 on dist-test-slave-phbh"
server_key: "ad92fbe446ca44b211e41ac81aac2264"
server_key_iv: "a57ffc537356f8771a454206d9083de9"
server_key_version: "encryptionkey@0"
I20260926 19:33:56.499303 14550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "7c1447da7f0040fda9bab90ce1a972fa"
format_stamp: "Formatted at 2026-09-26 19:33:56 on dist-test-slave-phbh"
server_key: "ad92fbe446ca44b211e41ac81aac2264"
server_key_iv: "a57ffc537356f8771a454206d9083de9"
server_key_version: "encryptionkey@0"
I20260926 19:33:56.504313 14550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 19:33:56.508095 14565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:56.509495 14550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:33:56.509687 14550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "7c1447da7f0040fda9bab90ce1a972fa"
format_stamp: "Formatted at 2026-09-26 19:33:56 on dist-test-slave-phbh"
server_key: "ad92fbe446ca44b211e41ac81aac2264"
server_key_iv: "a57ffc537356f8771a454206d9083de9"
server_key_version: "encryptionkey@0"
I20260926 19:33:56.509922 14550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:56.540493 14550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:56.545488 14550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:56.545810 14550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:56.568290 14550 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:39141
I20260926 19:33:56.568310 14626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:39141 every 8 connection(s)
I20260926 19:33:56.569720 14550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:33:56.573499 14627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:56.573513 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 14550
I20260926 19:33:56.573711 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:33:56.574393 7007 external_mini_cluster.cc:1428] Setting key 87b8d1ce6ce06e983bce30e23086084e
I20260926 19:33:56.587967 14627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Bootstrap starting.
Sep 26 19:33:56 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:56.591167 14627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:56.592504 14627 log.cc:824] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:56.595721 14627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: No bootstrap required, opened a new log
I20260926 19:33:56.599077 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:56.576530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.6.215.254:39141)
0926 19:33:56.577362 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:56.577376 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:56.577433 (+ 57us) server_negotiation.cc:407] Connection header received
0926 19:33:56.580243 (+ 2810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:56.580273 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:56.580845 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:56.581316 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:56.582562 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:56.584556 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:56.585802 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:56.586840 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:56.587012 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:56.590819 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:56.590830 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:56.590848 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:56.590892 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:56.594580 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:56.595231 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:56.595238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:56.595246 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:56.595465 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:56.595790 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:56.595797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:56.595802 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:56.596489 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:56.596760 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:56.597298 (+ 538us) server_negotiation.cc:299] Negotiation successful
0926 19:33:56.597726 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":387,"threads_started":1}
I20260926 19:33:56.605024 14627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1447da7f0040fda9bab90ce1a972fa" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39141 } }
I20260926 19:33:56.605392 14627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:56.605465 14627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1447da7f0040fda9bab90ce1a972fa, State: Initialized, Role: FOLLOWER
I20260926 19:33:56.605996 14627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1447da7f0040fda9bab90ce1a972fa" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39141 } }
I20260926 19:33:56.606199 14627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:33:56.606293 14627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:33:56.606416 14627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:56.608445 14627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1447da7f0040fda9bab90ce1a972fa" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39141 } }
I20260926 19:33:56.608951 14627 leader_election.cc:302] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c1447da7f0040fda9bab90ce1a972fa; no voters:
I20260926 19:33:56.609473 14627 leader_election.cc:288] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:33:56.609570 14632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:56.610301 14632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [term 1 LEADER]: Becoming Leader. State: Replica: 7c1447da7f0040fda9bab90ce1a972fa, State: Running, Role: LEADER
I20260926 19:33:56.611011 14632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1447da7f0040fda9bab90ce1a972fa" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39141 } }
I20260926 19:33:56.611727 14627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:33:56.614003 14634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c1447da7f0040fda9bab90ce1a972fa. Latest consensus state: current_term: 1 leader_uuid: "7c1447da7f0040fda9bab90ce1a972fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1447da7f0040fda9bab90ce1a972fa" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39141 } } }
I20260926 19:33:56.614470 14634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:56.617816 14639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:33:56.617781 14633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c1447da7f0040fda9bab90ce1a972fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c1447da7f0040fda9bab90ce1a972fa" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39141 } } }
I20260926 19:33:56.618158 14633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa [sys.catalog]: This master's current role is: LEADER
I20260926 19:33:56.623172 14639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:33:56.632649 14639 catalog_manager.cc:1347] Generated new cluster ID: 59097ef08ccb423a8c0b6b6b34937fd1
I20260926 19:33:56.632805 14639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:33:56.649706 14639 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:33:56.651162 14639 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:33:56.662688 14639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Generated new TSK 0
I20260926 19:33:56.663617 14639 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab.
Entry for principal oryx/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab.
I20260926 19:33:56.732961 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39141
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab
--principal=oryx/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:57.067991 14653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:57.068284 14653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:57.068384 14653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:57.075866 14653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:57.075975 14653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:57.076058 14653 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:57.076138 14653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:33:57.086195 14653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab
--principal=oryx/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:39141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:57.087392 14653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:57.088661 14653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:57.101437 14661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:57.101480 14659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:57.101537 14658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:57.102419 14653 server_base.cc:1034] running on GCE node
I20260926 19:33:57.103075 14653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:57.103999 14653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:57.105214 14653 hybrid_clock.cc:648] HybridClock initialized: now 1790451237105144 us; error 34 us; skew 500 ppm
I20260926 19:33:57.105532 14653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:57 dist-test-slave-phbh krb5kdc[14537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451237, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:57.108697 14653 init.cc:377] Logged in from keytab as oryx/127.6.215.193@KRBTEST.COM (short username oryx)
I20260926 19:33:57.110170 14653 webserver.cc:460] Webserver started at http://127.6.215.193:33241/ using document root <none> and password file <none>
I20260926 19:33:57.110898 14653 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:57.111009 14653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:57.111340 14653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:57.114594 14653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "4dd2fabb62d84e9e9fafd9061dee4252"
format_stamp: "Formatted at 2026-09-26 19:33:57 on dist-test-slave-phbh"
server_key: "892ad721907f8f300f128f0aca2a32ef"
server_key_iv: "5c2fa71aeb65ac06e3f6ba67accb2929"
server_key_version: "encryptionkey@0"
I20260926 19:33:57.115370 14653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "4dd2fabb62d84e9e9fafd9061dee4252"
format_stamp: "Formatted at 2026-09-26 19:33:57 on dist-test-slave-phbh"
server_key: "892ad721907f8f300f128f0aca2a32ef"
server_key_iv: "5c2fa71aeb65ac06e3f6ba67accb2929"
server_key_version: "encryptionkey@0"
I20260926 19:33:57.120368 14653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:33:57.123839 14668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:57.125291 14653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:57.125463 14653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "4dd2fabb62d84e9e9fafd9061dee4252"
format_stamp: "Formatted at 2026-09-26 19:33:57 on dist-test-slave-phbh"
server_key: "892ad721907f8f300f128f0aca2a32ef"
server_key_iv: "5c2fa71aeb65ac06e3f6ba67accb2929"
server_key_version: "encryptionkey@0"
I20260926 19:33:57.125675 14653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:57.139654 14653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:57.143621 14653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:57.143927 14653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:57.146689 14653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:57.146798 14653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:57.146962 14653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:57.147053 14653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:57 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451237, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:57.171800 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.147391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:39233 (local address 127.6.215.254:39141)
0926 19:33:57.147906 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:57.147913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:57.149073 (+ 1160us) server_negotiation.cc:407] Connection header received
0926 19:33:57.151300 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:57.151306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:57.151397 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:57.151522 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:57.154222 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.155159 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.157721 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.158339 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.158467 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.162995 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:57.163000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:57.163003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:57.163034 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:57.166525 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.167563 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.167567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.167570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.167676 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.168395 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.168403 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.168407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.168878 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:57.169050 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:57.170931 (+ 1881us) server_negotiation.cc:299] Negotiation successful
0926 19:33:57.171241 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":290,"threads_started":1}
I20260926 19:33:57.172323 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.148019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39141 (local address 127.6.215.193:39233)
0926 19:33:57.148896 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:57.148950 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:57.151023 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:57.151893 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:57.151911 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:57.152960 (+ 1049us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:57.153991 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.154023 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.155390 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.155397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.157579 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.157590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.158585 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.158595 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.158959 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.159688 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:57.159726 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:57.162756 (+ 3030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:57.166690 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.166701 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.166726 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.167386 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.167864 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.167872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.167882 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.168219 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.169136 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:57.169147 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:57.169935 (+ 788us) client_negotiation.cc:769] Sending connection context
0926 19:33:57.170348 (+ 413us) client_negotiation.cc:240] Negotiation successful
0926 19:33:57.171098 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":382,"threads_started":1}
I20260926 19:33:57.189930 14653 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:34639
I20260926 19:33:57.189962 14788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:34639 every 8 connection(s)
I20260926 19:33:57.191540 14653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:33:57.193638 14789 heartbeater.cc:347] Connected to a master server at 127.6.215.254:39141
I20260926 19:33:57.193897 14789 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:57.194710 14789 heartbeater.cc:511] Master 127.6.215.254:39141 requested a full tablet report, sending...
I20260926 19:33:57.196359 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.180494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:36183 (local address 127.6.215.254:39141)
0926 19:33:57.180713 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:57.180722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:57.181413 (+ 691us) server_negotiation.cc:407] Connection header received
0926 19:33:57.181594 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:57.181602 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:57.181703 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:57.181835 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:57.183768 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.185046 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.186349 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.187130 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.187264 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.190139 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:57.190147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:57.190152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:57.190189 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:57.193433 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.194326 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.194330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.194332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.194445 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.194907 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.194913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.194916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.195205 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:57.195359 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:57.195785 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 19:33:57.196000 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 19:33:57.196401 14764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.180536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39141 (local address 127.6.215.193:36183)
0926 19:33:57.181303 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:57.181323 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:57.181459 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:57.182057 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:57.182065 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:57.182927 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:57.183535 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.183552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.185187 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.185191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.186201 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.186212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.187589 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.187602 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.187879 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.188531 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:57.188556 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:57.189893 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:57.193645 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.193652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.193658 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.194214 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.194559 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.194566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.194571 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.194731 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.195349 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:57.195353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:57.195526 (+ 173us) client_negotiation.cc:769] Sending connection context
0926 19:33:57.195673 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 19:33:57.195968 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":436,"threads_started":1}
I20260926 19:33:57.196925 14582 ts_manager.cc:194] Registered new tserver with Master: 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639)
I20260926 19:33:57.197495 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 14653
I20260926 19:33:57.197688 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:33:57.198503 7007 external_mini_cluster.cc:1428] Setting key a300fd0bba55a51a2538a520e00018c5
I20260926 19:33:57.199002 14582 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:39233
WARNING: no policy specified for oryx/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab.
Entry for principal oryx/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab.
I20260926 19:33:57.245385 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39141
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab
--principal=oryx/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:57.567113 14793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:57.567392 14793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:57.567517 14793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:57.574965 14793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:57.575086 14793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:57.575269 14793 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:57.575388 14793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:33:57.585590 14793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab
--principal=oryx/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:39141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:57.586838 14793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:57.588200 14793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:57.600890 14798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:57.600931 14799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:57.600957 14801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:57.601799 14793 server_base.cc:1034] running on GCE node
I20260926 19:33:57.602386 14793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:57.603288 14793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:57.604532 14793 hybrid_clock.cc:648] HybridClock initialized: now 1790451237604460 us; error 55 us; skew 500 ppm
I20260926 19:33:57.604898 14793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:57 dist-test-slave-phbh krb5kdc[14537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451237, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:57.608049 14793 init.cc:377] Logged in from keytab as oryx/127.6.215.194@KRBTEST.COM (short username oryx)
I20260926 19:33:57.609596 14793 webserver.cc:460] Webserver started at http://127.6.215.194:38855/ using document root <none> and password file <none>
I20260926 19:33:57.610293 14793 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:57.610422 14793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:57.610739 14793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:57.613852 14793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "9d723907ea6e4bcba154190572fdca9e"
format_stamp: "Formatted at 2026-09-26 19:33:57 on dist-test-slave-phbh"
server_key: "d336c0032e74a323cd6c65efb84243d4"
server_key_iv: "0de869b8c7ff04e5e3921e702f296747"
server_key_version: "encryptionkey@0"
I20260926 19:33:57.614753 14793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "9d723907ea6e4bcba154190572fdca9e"
format_stamp: "Formatted at 2026-09-26 19:33:57 on dist-test-slave-phbh"
server_key: "d336c0032e74a323cd6c65efb84243d4"
server_key_iv: "0de869b8c7ff04e5e3921e702f296747"
server_key_version: "encryptionkey@0"
I20260926 19:33:57.619870 14793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:33:57.624097 14808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:57.625654 14793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:57.625820 14793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "9d723907ea6e4bcba154190572fdca9e"
format_stamp: "Formatted at 2026-09-26 19:33:57 on dist-test-slave-phbh"
server_key: "d336c0032e74a323cd6c65efb84243d4"
server_key_iv: "0de869b8c7ff04e5e3921e702f296747"
server_key_version: "encryptionkey@0"
I20260926 19:33:57.626040 14793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:57.641887 14793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:57.646184 14793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:57.646457 14793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:57.649093 14793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:57.649199 14793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:57.649362 14793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:57.649438 14793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:57 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451237, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:57.672751 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.649887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:41183 (local address 127.6.215.254:39141)
0926 19:33:57.650068 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:57.650074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:57.651504 (+ 1430us) server_negotiation.cc:407] Connection header received
0926 19:33:57.654510 (+ 3006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:57.654519 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:57.654649 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:57.654768 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:57.657402 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.658407 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.660241 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.660902 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.661038 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.665354 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:57.665364 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:57.665372 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:57.665418 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:57.669178 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.670143 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.670148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.670156 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.670321 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.670995 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.671003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.671008 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.671316 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:57.671478 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:57.672188 (+ 710us) server_negotiation.cc:299] Negotiation successful
0926 19:33:57.672412 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:33:57.673914 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.650406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39141 (local address 127.6.215.194:41183)
0926 19:33:57.651297 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:57.651351 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:57.654096 (+ 2745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:57.655057 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:57.655070 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:57.656091 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:57.657166 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.657199 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.658619 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.658623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.660090 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.660102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.661154 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.661161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.661403 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.662145 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:57.662183 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:57.665087 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:57.669325 (+ 4238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.669332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.669344 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.670014 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.670460 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.670468 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.670474 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.670837 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.671450 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:57.671459 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:57.671810 (+ 351us) client_negotiation.cc:769] Sending connection context
0926 19:33:57.672069 (+ 259us) client_negotiation.cc:240] Negotiation successful
0926 19:33:57.672593 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":318,"threads_started":1}
I20260926 19:33:57.696022 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.679777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:51371 (local address 127.6.215.254:39141)
0926 19:33:57.680229 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:57.680238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:57.680795 (+ 557us) server_negotiation.cc:407] Connection header received
0926 19:33:57.681015 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:57.681024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:57.681130 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:57.681270 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:57.682224 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.683709 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.685241 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.686058 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.686200 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.689207 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:57.689212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:57.689215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:57.689244 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:57.692416 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.693430 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.693438 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.693445 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.693603 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:57.694017 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:57.694026 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:57.694030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:57.694540 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:57.694709 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:57.695482 (+ 773us) server_negotiation.cc:299] Negotiation successful
0926 19:33:57.695713 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260926 19:33:57.696331 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:57.679884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39141 (local address 127.6.215.194:51371)
0926 19:33:57.680637 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:57.680662 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:57.680846 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:57.681391 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:57.681395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:57.681825 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:57.682093 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.682101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.683875 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.683879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.685081 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:57.685093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:57.686271 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:57.686277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:57.686482 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:57.687200 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:57.687230 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:57.688958 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:57.692556 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.692562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.692570 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.693232 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.693706 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:57.693711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:57.693715 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:57.693851 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:57.694784 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:57.694791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:57.695080 (+ 289us) client_negotiation.cc:769] Sending connection context
0926 19:33:57.695321 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 19:33:57.695765 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":419,"threads_started":1}
I20260926 19:33:57.699577 14793 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:37619
I20260926 19:33:57.699723 14927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:37619 every 8 connection(s)
I20260926 19:33:57.700858 14793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:33:57.702651 14928 heartbeater.cc:347] Connected to a master server at 127.6.215.254:39141
I20260926 19:33:57.702826 14928 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:57.703444 14928 heartbeater.cc:511] Master 127.6.215.254:39141 requested a full tablet report, sending...
I20260926 19:33:57.704707 14581 ts_manager.cc:194] Registered new tserver with Master: 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619)
I20260926 19:33:57.705628 14581 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:41183
I20260926 19:33:57.706254 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 14793
I20260926 19:33:57.706432 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:33:57.706847 7007 external_mini_cluster.cc:1428] Setting key f91cea29045e8909e7464fc5926869fe
WARNING: no policy specified for oryx/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab.
Entry for principal oryx/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab.
I20260926 19:33:57.752195 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39141
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab
--principal=oryx/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:35879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:33:58.083916 14932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:33:58.084204 14932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:33:58.084355 14932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:33:58.091567 14932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:33:58.091672 14932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:33:58.091814 14932 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 19:33:58.091928 14932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:33:58.101698 14932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:35879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab
--principal=oryx/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.215.254:39141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:33:58.102967 14932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:33:58.104247 14932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:33:58.117481 14937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:58.117518 14940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:33:58.117432 14938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:33:58.118265 14932 server_base.cc:1034] running on GCE node
I20260926 19:33:58.118877 14932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:33:58.119824 14932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:33:58.121061 14932 hybrid_clock.cc:648] HybridClock initialized: now 1790451238120989 us; error 49 us; skew 500 ppm
I20260926 19:33:58.121378 14932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:33:58 dist-test-slave-phbh krb5kdc[14537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451238, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:33:58.124589 14932 init.cc:377] Logged in from keytab as oryx/127.6.215.195@KRBTEST.COM (short username oryx)
I20260926 19:33:58.126132 14932 webserver.cc:460] Webserver started at http://127.6.215.195:40107/ using document root <none> and password file <none>
I20260926 19:33:58.126829 14932 fs_manager.cc:359] Metadata directory not provided
I20260926 19:33:58.126955 14932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:33:58.127272 14932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:33:58.130456 14932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "7e69b2bc8c814323b9e4f0ee56278a37"
format_stamp: "Formatted at 2026-09-26 19:33:58 on dist-test-slave-phbh"
server_key: "52075e4a2474530e318c26c527400e8f"
server_key_iv: "0a03d56032ac0e67b407f3af57973537"
server_key_version: "encryptionkey@0"
I20260926 19:33:58.131273 14932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "7e69b2bc8c814323b9e4f0ee56278a37"
format_stamp: "Formatted at 2026-09-26 19:33:58 on dist-test-slave-phbh"
server_key: "52075e4a2474530e318c26c527400e8f"
server_key_iv: "0a03d56032ac0e67b407f3af57973537"
server_key_version: "encryptionkey@0"
I20260926 19:33:58.136376 14932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260926 19:33:58.140354 14947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:58.141894 14932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:33:58.142086 14932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "7e69b2bc8c814323b9e4f0ee56278a37"
format_stamp: "Formatted at 2026-09-26 19:33:58 on dist-test-slave-phbh"
server_key: "52075e4a2474530e318c26c527400e8f"
server_key_iv: "0a03d56032ac0e67b407f3af57973537"
server_key_version: "encryptionkey@0"
I20260926 19:33:58.142346 14932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:33:58.173082 14932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:33:58.178171 14932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:33:58.178447 14932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:33:58.181056 14932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:33:58.181134 14932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:33:58.181277 14932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:33:58.181349 14932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:33:58 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451238, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:58.202351 14789 heartbeater.cc:503] Master 127.6.215.254:39141 was elected leader, sending a full tablet report...
I20260926 19:33:58.206705 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:58.181926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:33365 (local address 127.6.215.254:39141)
0926 19:33:58.182083 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:58.182088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:58.183610 (+ 1522us) server_negotiation.cc:407] Connection header received
0926 19:33:58.185966 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:58.185972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:58.186069 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:58.186172 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:58.188983 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.190372 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.193590 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.194425 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.194580 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:58.199647 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:58.199653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:58.199658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:58.199697 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:58.202906 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:58.203988 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:58.203993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:58.204001 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:58.204129 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:58.204841 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:58.204846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:58.204852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:58.205169 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:58.205329 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:58.206120 (+ 791us) server_negotiation.cc:299] Negotiation successful
0926 19:33:58.206341 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 19:33:58.207970 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:58.182619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39141 (local address 127.6.215.195:33365)
0926 19:33:58.183446 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:58.183484 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:58.185603 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:58.186791 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:58.186808 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:58.187714 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:58.188715 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:58.188745 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.190599 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.190606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:58.193380 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:58.193401 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.194692 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.194699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:58.195017 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:58.196212 (+ 1195us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:58.196244 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:58.199406 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:58.203181 (+ 3775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:58.203190 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:58.203210 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:58.203854 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:58.204353 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:58.204358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:58.204361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:58.204647 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:58.205313 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:58.205321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:58.205702 (+ 381us) client_negotiation.cc:769] Sending connection context
0926 19:33:58.205988 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 19:33:58.206552 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":393,"threads_started":1}
I20260926 19:33:58.224072 14932 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:34843
I20260926 19:33:58.224094 15066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:34843 every 8 connection(s)
I20260926 19:33:58.225381 14932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:33:58.227478 15067 heartbeater.cc:347] Connected to a master server at 127.6.215.254:39141
I20260926 19:33:58.227705 15067 heartbeater.cc:464] Registering TS with master...
I20260926 19:33:58.228356 15067 heartbeater.cc:511] Master 127.6.215.254:39141 requested a full tablet report, sending...
I20260926 19:33:58.229391 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:58.215049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:35053 (local address 127.6.215.254:39141)
0926 19:33:58.215233 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:58.215239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:58.216309 (+ 1070us) server_negotiation.cc:407] Connection header received
0926 19:33:58.216501 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:58.216505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:58.216566 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:58.216678 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:58.217711 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.218641 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.220230 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.220978 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.221275 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:58.223421 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:58.223427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:58.223435 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:58.223474 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:58.226430 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:58.227274 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:58.227279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:58.227282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:58.227456 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:58.227834 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:58.227841 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:58.227845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:58.228120 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:58.228273 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:58.228750 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 19:33:58.228988 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:33:58.229437 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:58.215277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39141 (local address 127.6.215.195:35053)
0926 19:33:58.216166 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:58.216192 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:58.216381 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:58.216717 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:58.216721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:58.217238 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:33:58.217565 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:58.217575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.218807 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.218811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:58.220083 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:58.220098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.221266 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.221274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:58.221477 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:58.222452 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:33:58.222470 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:33:58.223213 (+ 743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:33:58.226585 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:58.226590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:58.226595 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:58.227031 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:58.227572 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:33:58.227577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:33:58.227583 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:33:58.227722 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:33:58.228250 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:33:58.228255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:33:58.228486 (+ 231us) client_negotiation.cc:769] Sending connection context
0926 19:33:58.228657 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 19:33:58.229004 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":539,"threads_started":1}
I20260926 19:33:58.229813 14581 ts_manager.cc:194] Registered new tserver with Master: 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:33:58.230868 14581 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:33365
I20260926 19:33:58.233863 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 14932
I20260926 19:33:58.234004 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:33:58.234396 7007 external_mini_cluster.cc:1428] Setting key 782d74600e5e79241ba60cef0d6a24a5
I20260926 19:33:58.237257 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 19:33:58 dist-test-slave-phbh krb5kdc[14537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451238, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 19:33:58 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:33:58 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:33:58.708289 14928 heartbeater.cc:503] Master 127.6.215.254:39141 was elected leader, sending a full tablet report...
I20260926 19:33:59.233340 15067 heartbeater.cc:503] Master 127.6.215.254:39141 was elected leader, sending a full tablet report...
W20260926 19:33:59.349086 15075 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.068s user 0.000s sys 0.015s
W20260926 19:33:59.350258 14677 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:33:58.275344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.6.215.254:39141)
0926 19:33:58.275584 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:58.275593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:58.275980 (+ 387us) server_negotiation.cc:407] Connection header received
0926 19:33:58.276114 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:58.276118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:58.276181 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:58.276275 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:58.277691 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.278537 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.279692 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:58.280075 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:58.280180 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.349934 (+1069754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54562: BlockingRecv error: recv got EOF from 127.0.0.1:54562 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 26 19:33:59 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451238, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:33:59.371510 14677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.354296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.6.215.254:39141)
0926 19:33:59.354532 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:59.354541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:59.354777 (+ 236us) server_negotiation.cc:407] Connection header received
0926 19:33:59.355005 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:59.355010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:59.355089 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:59.355193 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:33:59.356376 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.357158 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.358367 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.358791 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.358918 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.366513 (+ 7595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:33:59.366519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:33:59.366522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:33:59.366558 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:33:59.369062 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:59.369698 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:59.369704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:59.369708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:59.369826 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:33:59.370185 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:33:59.370191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:33:59.370195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:33:59.370460 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:33:59.370770 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:59.370975 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 19:33:59.371202 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:33:59.375713 14581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:59.382716 14581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:59.424480 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.395323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.6.215.195:34843)
0926 19:33:59.413499 (+ 18176us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:59.413510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:59.413547 (+ 37us) server_negotiation.cc:407] Connection header received
0926 19:33:59.414031 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:59.414047 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:59.414356 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:59.414570 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:59.416116 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.419061 (+ 2945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.421889 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.423145 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.423281 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.423512 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:59.423958 (+ 446us) server_negotiation.cc:299] Negotiation successful
0926 19:33:59.424164 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17991,"thread_start_us":18258,"threads_started":1}
I20260926 19:33:59.424579 15089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.395074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34843 (local address 127.0.0.1:44692)
0926 19:33:59.405216 (+ 10142us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:59.405238 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:59.405390 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:59.414828 (+ 9438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:59.414838 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:59.414875 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:59.415328 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.415343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.419219 (+ 3876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.419225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.421688 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.421705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.423446 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.423454 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.423673 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.423737 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 19:33:59.423897 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 19:33:59.424164 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9990,"thread_start_us":9828,"threads_started":1}
I20260926 19:33:59.427580 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.394861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.6.215.194:37619)
0926 19:33:59.419642 (+ 24781us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:59.419648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:59.419672 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:33:59.419745 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:59.419750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:59.419921 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:59.420982 (+ 1061us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:59.421191 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.422696 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.425378 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.426560 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.426703 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.426913 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:59.427151 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 19:33:59.427310 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24631,"thread_start_us":24497,"threads_started":1}
I20260926 19:33:59.427599 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.394659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37619 (local address 127.0.0.1:49734)
0926 19:33:59.404382 (+ 9723us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:59.404446 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:59.404688 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:59.420169 (+ 15481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:59.420179 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:59.420216 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:59.420737 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.420748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.422865 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.422873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.425227 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.425238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.426780 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.426789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.426938 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.426969 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:33:59.427072 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 19:33:59.427209 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9342,"mutex_wait_us":143,"spinlock_wait_cycles":7296,"thread_start_us":22219,"threads_started":1}
I20260926 19:33:59.429384 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.401127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34639 (local address 127.0.0.1:45778)
0926 19:33:59.418570 (+ 17443us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:59.418590 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:59.418720 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:59.420022 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:59.420028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:59.420060 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:59.420631 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.420650 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.422795 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.422800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.425819 (+ 3019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.425834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.428443 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.428449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.428668 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.428690 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 19:33:59.428884 (+ 194us) client_negotiation.cc:240] Negotiation successful
0926 19:33:59.429026 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17296,"mutex_wait_us":80,"spinlock_wait_cycles":3840,"thread_start_us":2685,"threads_started":1}
I20260926 19:33:59.428982 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 6da56a98ba8a464481c315d442d6e137 (DEFAULT_TABLE table=test-table [id=b457099907b34ebf9c49c8f20fcf67f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:59.430447 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da56a98ba8a464481c315d442d6e137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:59.431173 14858 tablet_service.cc:1467] Processing CreateTablet for tablet 6da56a98ba8a464481c315d442d6e137 (DEFAULT_TABLE table=test-table [id=b457099907b34ebf9c49c8f20fcf67f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:59.432214 14858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da56a98ba8a464481c315d442d6e137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:59.433524 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.394675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.6.215.193:34639)
0926 19:33:59.419240 (+ 24565us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:59.419251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:59.419295 (+ 44us) server_negotiation.cc:407] Connection header received
0926 19:33:59.419462 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:59.419471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:59.419776 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:59.420016 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:59.421101 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.422650 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.426013 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.428227 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.428389 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.428596 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:59.433056 (+ 4460us) server_negotiation.cc:299] Negotiation successful
0926 19:33:59.433236 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24384,"thread_start_us":24246,"threads_started":1}
I20260926 19:33:59.436820 14719 tablet_service.cc:1467] Processing CreateTablet for tablet 6da56a98ba8a464481c315d442d6e137 (DEFAULT_TABLE table=test-table [id=b457099907b34ebf9c49c8f20fcf67f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:59.438161 14719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6da56a98ba8a464481c315d442d6e137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:59.443624 15092 tablet_bootstrap.cc:492] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Bootstrap starting.
I20260926 19:33:59.446978 15092 tablet_bootstrap.cc:654] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:59.448113 15092 log.cc:824] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:59.449646 15093 tablet_bootstrap.cc:492] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Bootstrap starting.
I20260926 19:33:59.453087 15092 tablet_bootstrap.cc:492] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: No bootstrap required, opened a new log
I20260926 19:33:59.453147 15095 tablet_bootstrap.cc:492] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Bootstrap starting.
I20260926 19:33:59.453698 15093 tablet_bootstrap.cc:654] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:59.454317 15092 ts_tablet_manager.cc:1397] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260926 19:33:59.454984 15093 log.cc:824] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:59.456449 15095 tablet_bootstrap.cc:654] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:59.457461 15095 log.cc:824] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Log is configured to *not* fsync() on all Append() calls
I20260926 19:33:59.458082 15093 tablet_bootstrap.cc:492] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: No bootstrap required, opened a new log
I20260926 19:33:59.458374 15093 ts_tablet_manager.cc:1397] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 19:33:59.459863 15095 tablet_bootstrap.cc:492] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: No bootstrap required, opened a new log
I20260926 19:33:59.460122 15095 ts_tablet_manager.cc:1397] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260926 19:33:59.464690 15092 raft_consensus.cc:348] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.465173 15092 raft_consensus.cc:374] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:59.465276 15092 raft_consensus.cc:729] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d723907ea6e4bcba154190572fdca9e, State: Initialized, Role: FOLLOWER
I20260926 19:33:59.465920 15092 consensus_queue.cc:260] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.467221 15095 raft_consensus.cc:348] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.467669 15095 raft_consensus.cc:374] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:59.467819 15095 raft_consensus.cc:729] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Initialized, Role: FOLLOWER
I20260926 19:33:59.468546 15095 consensus_queue.cc:260] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.469173 15092 ts_tablet_manager.cc:1428] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260926 19:33:59.469296 15093 raft_consensus.cc:348] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.470629 15095 ts_tablet_manager.cc:1428] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260926 19:33:59.471094 15093 raft_consensus.cc:374] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:59.471213 15093 raft_consensus.cc:729] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e69b2bc8c814323b9e4f0ee56278a37, State: Initialized, Role: FOLLOWER
I20260926 19:33:59.471915 15093 consensus_queue.cc:260] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.473661 15093 ts_tablet_manager.cc:1428] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
W20260926 19:33:59.477734 15068 tablet.cc:2369] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:59.604024 15099 raft_consensus.cc:482] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:59.604364 15099 raft_consensus.cc:504] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.606057 15099 leader_election.cc:288] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:33:59.613009 15090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.606621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:40961 (local address 127.6.215.195:34843)
0926 19:33:59.606913 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:59.606921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:59.607558 (+ 637us) server_negotiation.cc:407] Connection header received
0926 19:33:59.607734 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:59.607738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:59.607793 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:59.607899 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:59.608535 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.609296 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.611244 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.611968 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.612176 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.612261 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:59.612609 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 19:33:59.612738 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:33:59.613024 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.606509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:35017 (local address 127.6.215.194:37619)
0926 19:33:59.606719 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:59.606727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:59.607422 (+ 695us) server_negotiation.cc:407] Connection header received
0926 19:33:59.607693 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:59.607698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:59.607766 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:59.607832 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:33:59.608457 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.609276 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.611219 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.611998 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.612179 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.612273 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:59.612641 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 19:33:59.612789 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 19:33:59.613253 15102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.606641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34843 (local address 127.6.215.193:40961)
0926 19:33:59.607454 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:59.607468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:59.607574 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:59.607933 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:59.607938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:59.607967 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:59.608350 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.608360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.609453 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.609457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.611063 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.611076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.612315 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.612325 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.612456 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.612479 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 19:33:59.612553 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 19:33:59.612705 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":27,"thread_start_us":512,"threads_started":1}
I20260926 19:33:59.613329 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.606528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37619 (local address 127.6.215.193:35017)
0926 19:33:59.607298 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0926 19:33:59.607316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:33:59.607508 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:33:59.607874 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:33:59.607883 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:33:59.607907 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:33:59.608310 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.608323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.609398 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.609403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.611065 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:33:59.611077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.612302 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.612309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:33:59.612470 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.612493 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 19:33:59.612598 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 19:33:59.612728 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":44,"thread_start_us":416,"threads_started":1}
I20260926 19:33:59.614436 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da56a98ba8a464481c315d442d6e137" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" is_pre_election: true
I20260926 19:33:59.614439 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da56a98ba8a464481c315d442d6e137" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e" is_pre_election: true
I20260926 19:33:59.614913 15017 raft_consensus.cc:2454] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 0.
I20260926 19:33:59.614923 14882 raft_consensus.cc:2454] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 0.
I20260926 19:33:59.615588 14671 leader_election.cc:302] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 7e69b2bc8c814323b9e4f0ee56278a37, 9d723907ea6e4bcba154190572fdca9e; no voters:
I20260926 19:33:59.615971 15099 raft_consensus.cc:2781] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:59.616117 15099 raft_consensus.cc:482] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:59.616225 15099 raft_consensus.cc:3037] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:59.618417 15099 raft_consensus.cc:504] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.619020 15099 leader_election.cc:288] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 election: Requested vote from peers 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:33:59.619546 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da56a98ba8a464481c315d442d6e137" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e"
I20260926 19:33:59.619546 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6da56a98ba8a464481c315d442d6e137" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37"
I20260926 19:33:59.619750 14882 raft_consensus.cc:3037] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:59.619750 15017 raft_consensus.cc:3037] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:59.622915 15017 raft_consensus.cc:2454] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 1.
I20260926 19:33:59.622915 14882 raft_consensus.cc:2454] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 1.
I20260926 19:33:59.623355 14670 leader_election.cc:302] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 9d723907ea6e4bcba154190572fdca9e; no voters:
I20260926 19:33:59.623703 15099 raft_consensus.cc:2781] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:59.624176 15099 raft_consensus.cc:686] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 LEADER]: Becoming Leader. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Running, Role: LEADER
I20260926 19:33:59.624704 15099 consensus_queue.cc:237] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.629264 14581 catalog_manager.cc:5495] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193). New cstate: current_term: 1 leader_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } health_report { overall_health: UNKNOWN } } }
I20260926 19:33:59.651571 15086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:33:59.646408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45782 (local address 127.6.215.193:34639)
0926 19:33:59.646581 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 19:33:59.646586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:33:59.646724 (+ 138us) server_negotiation.cc:407] Connection header received
0926 19:33:59.646952 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:33:59.646956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:33:59.647034 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:33:59.647198 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:33:59.647521 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.648173 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.649621 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:33:59.650012 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:33:59.650104 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:33:59.650483 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:33:59.651019 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:33:59.651126 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 19:33:59.651253 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 19:33:59.651339 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 19:33:59.659405 14882 raft_consensus.cc:1261] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Refusing update from remote peer 4dd2fabb62d84e9e9fafd9061dee4252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:59.659415 15017 raft_consensus.cc:1261] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Refusing update from remote peer 4dd2fabb62d84e9e9fafd9061dee4252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:33:59.660343 15103 consensus_queue.cc:1035] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:59.660681 15099 consensus_queue.cc:1035] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:33:59.672358 15106 mvcc.cc:205] Tried to move back new op lower bound from 7333688277636018176 to 7333688277507223552. Current Snapshot: MvccSnapshot[applied={T|T < 7333688277636018176}]
I20260926 19:33:59.686910 14582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:59.687170 14582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:33:59.690510 14582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Sending DeleteTablet for 3 replicas of tablet 6da56a98ba8a464481c315d442d6e137
I20260926 19:33:59.691630 14719 tablet_service.cc:1514] Processing DeleteTablet for tablet 6da56a98ba8a464481c315d442d6e137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:59 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45778
I20260926 19:33:59.691843 14858 tablet_service.cc:1514] Processing DeleteTablet for tablet 6da56a98ba8a464481c315d442d6e137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:59 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:49734
I20260926 19:33:59.692540 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 6da56a98ba8a464481c315d442d6e137 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:33:59 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:44692
I20260926 19:33:59.693173 15115 tablet_replica.cc:330] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: stopping tablet replica
I20260926 19:33:59.692906 14582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:33:59.693718 14582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:33:59.693759 15115 raft_consensus.cc:2229] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:33:59.694483 15115 raft_consensus.cc:2258] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:59.695255 15117 tablet_replica.cc:330] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: stopping tablet replica
I20260926 19:33:59.695664 15117 raft_consensus.cc:2229] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:59.696163 15117 raft_consensus.cc:2258] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:59.698002 15115 ts_tablet_manager.cc:1905] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:59.698102 15117 ts_tablet_manager.cc:1905] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:59.703145 14719 tablet_service.cc:1467] Processing CreateTablet for tablet 62575f61d847402c824b02c7bd4a9ae2 (DEFAULT_TABLE table=test-table [id=b635ce66389845db994440a9f513fc16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:59.703154 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 62575f61d847402c824b02c7bd4a9ae2 (DEFAULT_TABLE table=test-table [id=b635ce66389845db994440a9f513fc16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:59.703577 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62575f61d847402c824b02c7bd4a9ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:59.703568 14719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62575f61d847402c824b02c7bd4a9ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:59.704954 14857 tablet_service.cc:1467] Processing CreateTablet for tablet 62575f61d847402c824b02c7bd4a9ae2 (DEFAULT_TABLE table=test-table [id=b635ce66389845db994440a9f513fc16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:33:59.705372 15117 ts_tablet_manager.cc:1918] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:59.705502 15117 log.cc:1196] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/6da56a98ba8a464481c315d442d6e137
I20260926 19:33:59.705508 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62575f61d847402c824b02c7bd4a9ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:33:59.705991 15117 ts_tablet_manager.cc:1939] T 6da56a98ba8a464481c315d442d6e137 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting consensus metadata
I20260926 19:33:59.710250 14568 catalog_manager.cc:4841] TS 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843): tablet 6da56a98ba8a464481c315d442d6e137 (table test-table [id=b457099907b34ebf9c49c8f20fcf67f7]) successfully deleted
I20260926 19:33:59.712452 15095 tablet_bootstrap.cc:492] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Bootstrap starting.
I20260926 19:33:59.714143 15095 tablet_bootstrap.cc:654] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:59.720852 15092 tablet_bootstrap.cc:492] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Bootstrap starting.
I20260926 19:33:59.721578 15116 tablet_replica.cc:330] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: stopping tablet replica
I20260926 19:33:59.722045 15116 raft_consensus.cc:2229] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:33:59.722589 15092 tablet_bootstrap.cc:654] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:59.722868 15093 tablet_bootstrap.cc:492] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: Bootstrap starting.
I20260926 19:33:59.724889 15093 tablet_bootstrap.cc:654] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: Neither blocks nor log segments found. Creating new log.
I20260926 19:33:59.734567 15115 ts_tablet_manager.cc:1918] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:59.734817 15115 log.cc:1196] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/6da56a98ba8a464481c315d442d6e137
I20260926 19:33:59.735507 15115 ts_tablet_manager.cc:1939] T 6da56a98ba8a464481c315d442d6e137 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting consensus metadata
I20260926 19:33:59.722586 15116 raft_consensus.cc:2258] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:33:59.739775 15116 ts_tablet_manager.cc:1905] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:33:59.742934 15093 tablet_bootstrap.cc:492] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: No bootstrap required, opened a new log
I20260926 19:33:59.742987 14569 catalog_manager.cc:4841] TS 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639): tablet 6da56a98ba8a464481c315d442d6e137 (table test-table [id=b457099907b34ebf9c49c8f20fcf67f7]) successfully deleted
I20260926 19:33:59.743155 15093 ts_tablet_manager.cc:1397] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.005s
I20260926 19:33:59.744022 15095 tablet_bootstrap.cc:492] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: No bootstrap required, opened a new log
I20260926 19:33:59.744180 15095 ts_tablet_manager.cc:1397] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.008s
I20260926 19:33:59.744050 15093 raft_consensus.cc:348] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.744469 15093 raft_consensus.cc:374] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:59.744603 15093 raft_consensus.cc:729] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e69b2bc8c814323b9e4f0ee56278a37, State: Initialized, Role: FOLLOWER
I20260926 19:33:59.745002 15093 consensus_queue.cc:260] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.745101 15095 raft_consensus.cc:348] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.745348 15095 raft_consensus.cc:374] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:59.745430 15092 tablet_bootstrap.cc:492] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: No bootstrap required, opened a new log
I20260926 19:33:59.745460 15095 raft_consensus.cc:729] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Initialized, Role: FOLLOWER
I20260926 19:33:59.745565 15092 ts_tablet_manager.cc:1397] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.009s
I20260926 19:33:59.745771 15095 consensus_queue.cc:260] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.746320 15092 raft_consensus.cc:348] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.746518 15092 raft_consensus.cc:374] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:33:59.746603 15092 raft_consensus.cc:729] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d723907ea6e4bcba154190572fdca9e, State: Initialized, Role: FOLLOWER
I20260926 19:33:59.746869 15092 consensus_queue.cc:260] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.748370 15093 ts_tablet_manager.cc:1428] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260926 19:33:59.748377 15095 ts_tablet_manager.cc:1428] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 19:33:59.749226 15092 ts_tablet_manager.cc:1428] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260926 19:33:59.752820 15116 ts_tablet_manager.cc:1918] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:33:59.752997 15116 log.cc:1196] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/6da56a98ba8a464481c315d442d6e137
I20260926 19:33:59.753448 15116 ts_tablet_manager.cc:1939] T 6da56a98ba8a464481c315d442d6e137 P 9d723907ea6e4bcba154190572fdca9e: Deleting consensus metadata
I20260926 19:33:59.754853 14567 catalog_manager.cc:4841] TS 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619): tablet 6da56a98ba8a464481c315d442d6e137 (table test-table [id=b457099907b34ebf9c49c8f20fcf67f7]) successfully deleted
W20260926 19:33:59.945181 14790 tablet.cc:2369] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:33:59.960120 14929 tablet.cc:2369] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:33:59.969525 15099 raft_consensus.cc:482] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:33:59.969776 15099 raft_consensus.cc:504] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.970563 15099 leader_election.cc:288] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:33:59.971125 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575f61d847402c824b02c7bd4a9ae2" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e" is_pre_election: true
I20260926 19:33:59.971155 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575f61d847402c824b02c7bd4a9ae2" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" is_pre_election: true
I20260926 19:33:59.971377 14882 raft_consensus.cc:2454] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 0.
I20260926 19:33:59.971429 15017 raft_consensus.cc:2454] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 0.
I20260926 19:33:59.971861 14671 leader_election.cc:302] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 7e69b2bc8c814323b9e4f0ee56278a37; no voters:
I20260926 19:33:59.972142 15099 raft_consensus.cc:2781] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:33:59.972285 15099 raft_consensus.cc:482] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:33:59.972409 15099 raft_consensus.cc:3037] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:59.974725 15099 raft_consensus.cc:504] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.975382 15099 leader_election.cc:288] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 election: Requested vote from peers 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:33:59.975910 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575f61d847402c824b02c7bd4a9ae2" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e"
I20260926 19:33:59.975994 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62575f61d847402c824b02c7bd4a9ae2" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37"
I20260926 19:33:59.976150 14882 raft_consensus.cc:3037] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:59.976244 15017 raft_consensus.cc:3037] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:33:59.978389 15017 raft_consensus.cc:2454] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 1.
I20260926 19:33:59.979068 14671 leader_election.cc:302] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 7e69b2bc8c814323b9e4f0ee56278a37; no voters:
I20260926 19:33:59.979218 14882 raft_consensus.cc:2454] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 1.
I20260926 19:33:59.979348 15099 raft_consensus.cc:2781] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:33:59.979574 15099 raft_consensus.cc:686] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 LEADER]: Becoming Leader. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Running, Role: LEADER
I20260926 19:33:59.979960 15099 consensus_queue.cc:237] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:33:59.982803 14581 catalog_manager.cc:5495] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193). New cstate: current_term: 1 leader_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } health_report { overall_health: UNKNOWN } } }
Sep 26 19:34:00 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451236, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.254@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:34:00.048678 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.035020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39141 (local address 127.0.0.1:54580)
0926 19:34:00.035707 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.035732 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.035903 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.036284 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.036288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.037080 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:00.037275 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.037285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.038281 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.038289 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.039227 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.039237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.039892 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.039898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.040069 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.040726 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:00.040743 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:00.043184 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:00.046155 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:00.046168 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:00.046180 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:00.046562 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:00.046908 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:00.046915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:00.046919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:00.047044 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:00.047592 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:00.047602 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:00.047758 (+ 156us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.047931 (+ 173us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.048216 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":350,"threads_started":1}
I20260926 19:34:00.048698 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.035088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.6.215.254:39141)
0926 19:34:00.036048 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.036052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.036067 (+ 15us) server_negotiation.cc:407] Connection header received
0926 19:34:00.036104 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.036107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.036156 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.036260 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:00.037421 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.038161 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.039363 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.039741 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.039873 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.043365 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:00.043370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:00.043374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:00.043399 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:00.045956 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:00.046678 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:00.046686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:00.046690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:00.046810 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:00.047141 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:00.047147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:00.047153 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:00.047471 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:00.047622 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.048013 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.048323 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":798,"thread_start_us":650,"threads_started":1}
I20260926 19:34:00.054307 14581 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:54580:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 19:34:00.055323 14581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:34:00.064200 14997 tablet_service.cc:1467] Processing CreateTablet for tablet c04ebef9ae514ff48f1b6f1eaa9d2012 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6ce5993421c34d768985d85163d50e23]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:34:00.064200 14858 tablet_service.cc:1467] Processing CreateTablet for tablet c04ebef9ae514ff48f1b6f1eaa9d2012 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6ce5993421c34d768985d85163d50e23]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:34:00.064364 14719 tablet_service.cc:1467] Processing CreateTablet for tablet c04ebef9ae514ff48f1b6f1eaa9d2012 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6ce5993421c34d768985d85163d50e23]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 19:34:00.064810 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04ebef9ae514ff48f1b6f1eaa9d2012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.064823 14858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04ebef9ae514ff48f1b6f1eaa9d2012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.065017 14719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04ebef9ae514ff48f1b6f1eaa9d2012. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.072252 15093 tablet_bootstrap.cc:492] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: Bootstrap starting.
I20260926 19:34:00.072693 15092 tablet_bootstrap.cc:492] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: Bootstrap starting.
I20260926 19:34:00.073395 15095 tablet_bootstrap.cc:492] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: Bootstrap starting.
I20260926 19:34:00.074054 15093 tablet_bootstrap.cc:654] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.074334 15092 tablet_bootstrap.cc:654] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.075253 15095 tablet_bootstrap.cc:654] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.084883 15093 tablet_bootstrap.cc:492] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: No bootstrap required, opened a new log
I20260926 19:34:00.085101 15092 tablet_bootstrap.cc:492] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: No bootstrap required, opened a new log
I20260926 19:34:00.085146 15093 ts_tablet_manager.cc:1397] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260926 19:34:00.085434 15092 ts_tablet_manager.cc:1397] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260926 19:34:00.085508 15095 tablet_bootstrap.cc:492] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: No bootstrap required, opened a new log
I20260926 19:34:00.085772 15095 ts_tablet_manager.cc:1397] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260926 19:34:00.085959 15093 raft_consensus.cc:348] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.086151 15092 raft_consensus.cc:348] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.086283 15093 raft_consensus.cc:374] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.086359 15093 raft_consensus.cc:729] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e69b2bc8c814323b9e4f0ee56278a37, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.086442 15092 raft_consensus.cc:374] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.086584 15092 raft_consensus.cc:729] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d723907ea6e4bcba154190572fdca9e, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.086910 15092 consensus_queue.cc:260] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.087078 15095 raft_consensus.cc:348] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.087399 15095 raft_consensus.cc:374] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.087522 15095 raft_consensus.cc:729] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.087422 15098 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.087802 15098 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:34:00.087703 15093 consensus_queue.cc:260] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.087841 15095 consensus_queue.cc:260] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.088239 15093 ts_tablet_manager.cc:1428] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260926 19:34:00.088299 15099 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.088656 15099 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:34:00.088646 15100 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.089003 15100 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:34:00.089797 15095 ts_tablet_manager.cc:1428] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260926 19:34:00.091082 15092 ts_tablet_manager.cc:1428] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260926 19:34:00.218174 15100 raft_consensus.cc:482] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:34:00.218401 15100 raft_consensus.cc:504] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.219823 15100 leader_election.cc:288] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639), 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619)
I20260926 19:34:00.241716 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.220227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:51751 (local address 127.6.215.193:34639)
0926 19:34:00.233850 (+ 13623us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.233860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.235584 (+ 1724us) server_negotiation.cc:407] Connection header received
0926 19:34:00.235798 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.235808 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.235904 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.236017 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:00.236710 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.237931 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.239977 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.240719 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.240926 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.240999 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.241306 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.241446 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13429,"thread_start_us":15745,"threads_started":1}
I20260926 19:34:00.241807 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.220131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34639 (local address 127.6.215.195:51751)
0926 19:34:00.235482 (+ 15351us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.235504 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.235656 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.236097 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.236101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.236125 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:00.236493 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.236506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.238084 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.238089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.239796 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.239810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.240925 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.240931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.241133 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.241166 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.241267 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.241401 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15234,"mutex_wait_us":1,"thread_start_us":16147,"threads_started":1}
I20260926 19:34:00.242713 14739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ebef9ae514ff48f1b6f1eaa9d2012" candidate_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" is_pre_election: true
I20260926 19:34:00.243130 14739 raft_consensus.cc:2454] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e69b2bc8c814323b9e4f0ee56278a37 in term 0.
I20260926 19:34:00.243757 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.220218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:58359 (local address 127.6.215.194:37619)
0926 19:34:00.236090 (+ 15872us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.236096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.236638 (+ 542us) server_negotiation.cc:407] Connection header received
0926 19:34:00.236832 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.236841 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.236895 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.236996 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:00.237573 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.238457 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.241157 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.242418 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.242628 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.242755 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.243291 (+ 536us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.243439 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15751,"thread_start_us":15644,"threads_started":1}
I20260926 19:34:00.243957 14951 leader_election.cc:302] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 7e69b2bc8c814323b9e4f0ee56278a37; no voters:
I20260926 19:34:00.244227 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.220144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37619 (local address 127.6.215.195:58359)
0926 19:34:00.236531 (+ 16387us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.236548 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.236685 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.237021 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.237026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.237050 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:00.237407 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.237415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.238594 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.238599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.241009 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.241022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.242920 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.242926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.243105 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.243144 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.243246 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.243374 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16261,"mutex_wait_us":28,"thread_start_us":15079,"threads_started":1}
I20260926 19:34:00.244400 15100 raft_consensus.cc:2781] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:34:00.244513 15100 raft_consensus.cc:482] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:34:00.244602 15100 raft_consensus.cc:3037] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.244882 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ebef9ae514ff48f1b6f1eaa9d2012" candidate_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e" is_pre_election: true
I20260926 19:34:00.245127 14882 raft_consensus.cc:2454] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e69b2bc8c814323b9e4f0ee56278a37 in term 0.
I20260926 19:34:00.247996 15100 raft_consensus.cc:504] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.249125 15100 leader_election.cc:288] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [CANDIDATE]: Term 1 election: Requested vote from peers 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639), 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619)
I20260926 19:34:00.249179 14739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ebef9ae514ff48f1b6f1eaa9d2012" candidate_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dd2fabb62d84e9e9fafd9061dee4252"
I20260926 19:34:00.249223 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04ebef9ae514ff48f1b6f1eaa9d2012" candidate_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e"
I20260926 19:34:00.249389 14739 raft_consensus.cc:3037] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.249424 14882 raft_consensus.cc:3037] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.252434 14882 raft_consensus.cc:2454] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e69b2bc8c814323b9e4f0ee56278a37 in term 1.
I20260926 19:34:00.252525 14739 raft_consensus.cc:2454] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e69b2bc8c814323b9e4f0ee56278a37 in term 1.
I20260926 19:34:00.252924 14949 leader_election.cc:302] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e69b2bc8c814323b9e4f0ee56278a37, 9d723907ea6e4bcba154190572fdca9e; no voters:
I20260926 19:34:00.253247 15100 raft_consensus.cc:2781] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:00.253621 15100 raft_consensus.cc:686] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 LEADER]: Becoming Leader. State: Replica: 7e69b2bc8c814323b9e4f0ee56278a37, State: Running, Role: LEADER
I20260926 19:34:00.254045 15100 consensus_queue.cc:237] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.256047 15136 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: TxnStatusTablet state changed. Reason: New leader 7e69b2bc8c814323b9e4f0ee56278a37. Latest consensus state: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.256422 15136 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:34:00.257090 15138 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 19:34:00.257292 15138 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 19:34:00.257671 14582 catalog_manager.cc:5495] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } health_report { overall_health: HEALTHY } } }
I20260926 19:34:00.278723 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.271727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34843 (local address 127.0.0.1:44696)
0926 19:34:00.271972 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.272000 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.272173 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.272960 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.272964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.272973 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:34:00.273126 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.273133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.274692 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.274696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.275862 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.275873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.276920 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.276925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.277065 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.277121 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.278267 (+ 1146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.278295 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.278374 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.278486 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 19:34:00.278930 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.271703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.6.215.195:34843)
0926 19:34:00.272481 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.272487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.272513 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:34:00.272598 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.272603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.272727 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.272919 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:00.273274 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.274534 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.275999 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.276726 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.276892 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.277340 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.278129 (+ 789us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.278293 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.278438 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.278527 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":465,"threads_started":1}
I20260926 19:34:00.283188 14882 raft_consensus.cc:1261] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Refusing update from remote peer 7e69b2bc8c814323b9e4f0ee56278a37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.283463 14739 raft_consensus.cc:1261] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Refusing update from remote peer 7e69b2bc8c814323b9e4f0ee56278a37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.283937 15136 consensus_queue.cc:1035] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:00.284252 15100 consensus_queue.cc:1035] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:00.290462 15099 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: TxnStatusTablet state changed. Reason: New leader 7e69b2bc8c814323b9e4f0ee56278a37. Latest consensus state: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.290875 15099 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:34:00.292049 15098 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: TxnStatusTablet state changed. Reason: New leader 7e69b2bc8c814323b9e4f0ee56278a37. Latest consensus state: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.292342 15098 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:34:00.293649 15142 mvcc.cc:205] Tried to move back new op lower bound from 7333688280193351680 to 7333688280084811776. Current Snapshot: MvccSnapshot[applied={T|T < 7333688280193351680}]
I20260926 19:34:00.296859 15099 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.297184 15099 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 4dd2fabb62d84e9e9fafd9061dee4252: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:34:00.300514 15100 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.300824 15100 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:34:00.301632 15098 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.301875 15098 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 9d723907ea6e4bcba154190572fdca9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 19:34:00.303303 15100 tablet_replica.cc:439] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } } }
I20260926 19:34:00.303534 15100 tablet_replica.cc:442] T c04ebef9ae514ff48f1b6f1eaa9d2012 P 7e69b2bc8c814323b9e4f0ee56278a37: This TxnStatusTablet replica's current role is: LEADER
I20260926 19:34:00.321348 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.315781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:34131 (local address 127.6.215.195:34843)
0926 19:34:00.315996 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.316005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.316456 (+ 451us) server_negotiation.cc:407] Connection header received
0926 19:34:00.316666 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.316671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.316748 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.316868 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:00.317512 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.318303 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.319633 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.320059 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.320187 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.320592 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.320692 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.320801 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.320959 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.321033 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 19:34:00.321429 15152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.315796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34843 (local address 127.6.215.193:34131)
0926 19:34:00.316340 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.316359 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.316535 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.316888 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.316893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.316905 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:34:00.317206 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.317214 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.318443 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.318450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.319488 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.319500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.320215 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.320221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.320357 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.320410 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.320835 (+ 425us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.320851 (+ 16us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.320938 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.321070 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":282,"threads_started":1}
I20260926 19:34:00.330940 15017 raft_consensus.cc:1261] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Refusing update from remote peer 4dd2fabb62d84e9e9fafd9061dee4252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.330989 14882 raft_consensus.cc:1261] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Refusing update from remote peer 4dd2fabb62d84e9e9fafd9061dee4252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.331492 15099 consensus_queue.cc:1035] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:00.331744 15103 consensus_queue.cc:1035] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:00.338414 15143 mvcc.cc:205] Tried to move back new op lower bound from 7333688280390144000 to 7333688278960652288. Current Snapshot: MvccSnapshot[applied={T|T < 7333688280390144000}]
I20260926 19:34:00.374106 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.367067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:38827 (local address 127.6.215.193:34639)
0926 19:34:00.367230 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.367235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.367949 (+ 714us) server_negotiation.cc:407] Connection header received
0926 19:34:00.368212 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.368219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.368304 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.368409 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:00.368940 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.370023 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.372014 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.372698 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.372875 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.373285 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.373451 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.373540 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.373740 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.373820 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 19:34:00.374256 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.367120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34639 (local address 127.6.215.195:38827)
0926 19:34:00.367842 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.367862 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.368025 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.368427 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.368430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.368440 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:34:00.368746 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.368754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.370159 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.370164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.371881 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.371893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.372925 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.372930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.373046 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.373116 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.373558 (+ 442us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.373569 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.373711 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.373856 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":368,"threads_started":1}
I20260926 19:34:00.419307 14581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:00.419592 14581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:00.422937 14581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Sending DeleteTablet for 3 replicas of tablet 62575f61d847402c824b02c7bd4a9ae2
I20260926 19:34:00.423754 14858 tablet_service.cc:1514] Processing DeleteTablet for tablet 62575f61d847402c824b02c7bd4a9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:00 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:49734
I20260926 19:34:00.424130 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 62575f61d847402c824b02c7bd4a9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:00 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:44692
I20260926 19:34:00.424209 14719 tablet_service.cc:1514] Processing DeleteTablet for tablet 62575f61d847402c824b02c7bd4a9ae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:00 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45778
I20260926 19:34:00.424458 15161 tablet_replica.cc:330] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: stopping tablet replica
I20260926 19:34:00.424708 15161 raft_consensus.cc:2229] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:00.425050 15161 raft_consensus.cc:2258] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:00.426192 15161 ts_tablet_manager.cc:1905] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:00.433602 15161 ts_tablet_manager.cc:1918] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:34:00.433826 15161 log.cc:1196] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/62575f61d847402c824b02c7bd4a9ae2
I20260926 19:34:00.434365 15161 ts_tablet_manager.cc:1939] T 62575f61d847402c824b02c7bd4a9ae2 P 9d723907ea6e4bcba154190572fdca9e: Deleting consensus metadata
I20260926 19:34:00.434434 15163 tablet_replica.cc:330] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: stopping tablet replica
I20260926 19:34:00.435278 15163 raft_consensus.cc:2229] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:00.435508 14567 catalog_manager.cc:4841] TS 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619): tablet 62575f61d847402c824b02c7bd4a9ae2 (table test-table [id=b635ce66389845db994440a9f513fc16]) successfully deleted
I20260926 19:34:00.435961 15163 raft_consensus.cc:2258] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:00.437345 15163 ts_tablet_manager.cc:1905] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:00.441200 15162 tablet_replica.cc:330] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: stopping tablet replica
I20260926 19:34:00.441464 15162 raft_consensus.cc:2229] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:34:00.441936 15162 raft_consensus.cc:2258] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:00.442986 15162 ts_tablet_manager.cc:1905] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:00.450769 15163 ts_tablet_manager.cc:1918] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:34:00.450963 15163 log.cc:1196] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/62575f61d847402c824b02c7bd4a9ae2
I20260926 19:34:00.450973 15162 ts_tablet_manager.cc:1918] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:34:00.451188 15162 log.cc:1196] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/62575f61d847402c824b02c7bd4a9ae2
I20260926 19:34:00.451489 15163 ts_tablet_manager.cc:1939] T 62575f61d847402c824b02c7bd4a9ae2 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting consensus metadata
I20260926 19:34:00.451701 15162 ts_tablet_manager.cc:1939] T 62575f61d847402c824b02c7bd4a9ae2 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting consensus metadata
I20260926 19:34:00.452687 14568 catalog_manager.cc:4841] TS 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843): tablet 62575f61d847402c824b02c7bd4a9ae2 (table test-table [id=b635ce66389845db994440a9f513fc16]) successfully deleted
I20260926 19:34:00.453521 14569 catalog_manager.cc:4841] TS 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639): tablet 62575f61d847402c824b02c7bd4a9ae2 (table test-table [id=b635ce66389845db994440a9f513fc16]) successfully deleted
Sep 26 19:34:00 dist-test-slave-phbh krb5kdc[14537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 19:34:00 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:00 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 19:34:00.493484 15128 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:00.481637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.6.215.254:39141)
0926 19:34:00.481864 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.481872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.482275 (+ 403us) server_negotiation.cc:407] Connection header received
0926 19:34:00.482420 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.482426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.482508 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.482627 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:00.484053 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.485180 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.486539 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.487202 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.487356 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.493211 (+ 5855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54586: BlockingRecv error: recv got EOF from 127.0.0.1:54586 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 26 19:34:00 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM
I20260926 19:34:00.513237 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.496463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.6.215.254:39141)
0926 19:34:00.496695 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.496702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.497033 (+ 331us) server_negotiation.cc:407] Connection header received
0926 19:34:00.497194 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.497198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.497261 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.497347 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:00.498930 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.499636 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.501475 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.501874 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.502006 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.506441 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:00.506448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:00.506453 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:00.506492 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:00.509951 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:00.510790 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:00.510799 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:00.510805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:00.510986 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:00.511473 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:00.511478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:00.511482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:00.511857 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:00.512165 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.512473 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.512819 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:34:00.517421 14581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:34:00.518112 14581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:34:00.525249 14719 tablet_service.cc:1467] Processing CreateTablet for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 (DEFAULT_TABLE table=test-table [id=5f6f3a9f5c724c57940fdb3c28db29cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:00.525668 14719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.529696 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 (DEFAULT_TABLE table=test-table [id=5f6f3a9f5c724c57940fdb3c28db29cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:00.529706 14858 tablet_service.cc:1467] Processing CreateTablet for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 (DEFAULT_TABLE table=test-table [id=5f6f3a9f5c724c57940fdb3c28db29cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:00.530182 14858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.530187 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.532021 15095 tablet_bootstrap.cc:492] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: Bootstrap starting.
I20260926 19:34:00.533546 15095 tablet_bootstrap.cc:654] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.537293 15092 tablet_bootstrap.cc:492] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Bootstrap starting.
I20260926 19:34:00.538218 15093 tablet_bootstrap.cc:492] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: Bootstrap starting.
I20260926 19:34:00.539072 15092 tablet_bootstrap.cc:654] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.540197 15093 tablet_bootstrap.cc:654] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.541400 15092 tablet_bootstrap.cc:492] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: No bootstrap required, opened a new log
I20260926 19:34:00.541528 15092 ts_tablet_manager.cc:1397] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260926 19:34:00.542253 15092 raft_consensus.cc:348] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.542497 15092 raft_consensus.cc:374] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.542584 15092 raft_consensus.cc:729] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d723907ea6e4bcba154190572fdca9e, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.542835 15092 consensus_queue.cc:260] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.543113 15093 tablet_bootstrap.cc:492] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: No bootstrap required, opened a new log
I20260926 19:34:00.543329 15093 ts_tablet_manager.cc:1397] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 19:34:00.543919 15092 ts_tablet_manager.cc:1428] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 19:34:00.544371 15093 raft_consensus.cc:348] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.544754 15093 raft_consensus.cc:374] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.544958 15093 raft_consensus.cc:729] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e69b2bc8c814323b9e4f0ee56278a37, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.545308 15093 consensus_queue.cc:260] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.548929 15093 ts_tablet_manager.cc:1428] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 19:34:00.550340 15095 tablet_bootstrap.cc:492] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: No bootstrap required, opened a new log
I20260926 19:34:00.550495 15095 ts_tablet_manager.cc:1397] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.011s
I20260926 19:34:00.551275 15095 raft_consensus.cc:348] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.551540 15095 raft_consensus.cc:374] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.551638 15095 raft_consensus.cc:729] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.551877 15095 consensus_queue.cc:260] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.552595 15095 ts_tablet_manager.cc:1428] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 19:34:00.602553 15099 raft_consensus.cc:482] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:34:00.602734 15099 raft_consensus.cc:504] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.603544 15099 leader_election.cc:288] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:34:00.604118 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31fc6ba31c4d68ac67ce28c9b4e990" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e" is_pre_election: true
I20260926 19:34:00.604166 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31fc6ba31c4d68ac67ce28c9b4e990" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" is_pre_election: true
I20260926 19:34:00.604347 14882 raft_consensus.cc:2454] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 0.
I20260926 19:34:00.604451 15017 raft_consensus.cc:2454] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 0.
I20260926 19:34:00.604786 14670 leader_election.cc:302] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 9d723907ea6e4bcba154190572fdca9e; no voters:
I20260926 19:34:00.605139 15099 raft_consensus.cc:2781] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:34:00.605232 15099 raft_consensus.cc:482] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:34:00.605361 15099 raft_consensus.cc:3037] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.607465 15099 raft_consensus.cc:504] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.608072 15099 leader_election.cc:288] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 election: Requested vote from peers 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:34:00.608528 14882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31fc6ba31c4d68ac67ce28c9b4e990" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d723907ea6e4bcba154190572fdca9e"
I20260926 19:34:00.608603 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31fc6ba31c4d68ac67ce28c9b4e990" candidate_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37"
I20260926 19:34:00.608798 14882 raft_consensus.cc:3037] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.608870 15017 raft_consensus.cc:3037] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.611340 15017 raft_consensus.cc:2454] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 1.
I20260926 19:34:00.611778 14671 leader_election.cc:302] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 7e69b2bc8c814323b9e4f0ee56278a37; no voters:
I20260926 19:34:00.611826 14882 raft_consensus.cc:2454] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dd2fabb62d84e9e9fafd9061dee4252 in term 1.
I20260926 19:34:00.612192 15099 raft_consensus.cc:2781] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:00.612429 15099 raft_consensus.cc:686] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 LEADER]: Becoming Leader. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Running, Role: LEADER
I20260926 19:34:00.612710 15099 consensus_queue.cc:237] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.615862 14582 catalog_manager.cc:5495] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193). New cstate: current_term: 1 leader_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } health_report { overall_health: UNKNOWN } } }
I20260926 19:34:00.647059 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.641461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.6.215.193:34639)
0926 19:34:00.641676 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.641683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.641704 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:34:00.641896 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.641903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.642005 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.642117 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:00.642422 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.643614 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.644990 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.645616 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.645757 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.646317 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.646449 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.646548 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.646677 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.646753 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 19:34:00.649883 15017 raft_consensus.cc:1261] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Refusing update from remote peer 4dd2fabb62d84e9e9fafd9061dee4252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.650017 14882 raft_consensus.cc:1261] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Refusing update from remote peer 4dd2fabb62d84e9e9fafd9061dee4252: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.650527 15159 consensus_queue.cc:1035] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:00.650867 15099 consensus_queue.cc:1035] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:00.662895 14581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:00.663156 14581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:00.667165 14581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Sending DeleteTablet for 3 replicas of tablet 9f31fc6ba31c4d68ac67ce28c9b4e990
I20260926 19:34:00.668118 14858 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:00 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:49734
I20260926 19:34:00.668255 14857 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:00 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:49734
I20260926 19:34:00.668397 15161 tablet_replica.cc:330] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: stopping tablet replica
I20260926 19:34:00.668463 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:00 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:44692
I20260926 19:34:00.668603 15161 raft_consensus.cc:2229] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:00.668759 15163 tablet_replica.cc:330] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: stopping tablet replica
I20260926 19:34:00.668792 15161 pending_rounds.cc:62] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Trying to abort 2 pending ops.
I20260926 19:34:00.669003 15163 raft_consensus.cc:2229] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:00.669225 15163 raft_consensus.cc:2258] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:00.668913 15161 pending_rounds.cc:68] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7333688281695285248 op_type: WRITE_OP write_request { tablet_id: "9f31fc6ba31c4d68ac67ce28c9b4e990" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\324\276\340\325\006\"8\n\ntest-admin\022*\n 5f6f3a9f5c724c57940fdb3c28db29cf\020\001\030\001 \001(\001" signature: "<redacted>"".\204$\304\017\360E\235rz\333\017N\342]\342HH\032\252\274\\\007#\236?\274\0301\311\263\037\333\356\334\243\204\255\351\200\311z\2331!\347\310\'g\026\3109\235\2239Z\357\230\231\235>\231Y\033" signing_key_seq_num: 0 } } request_id { client_id: "616404f4de694fed9f93b72a967f0d8b" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260926 19:34:00.669664 15161 pending_rounds.cc:68] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7333688281552351232 op_type: NO_OP noop_request { }
I20260926 19:34:00.669811 15161 raft_consensus.cc:2866] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260926 19:34:00.669888 14719 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:00 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45778
I20260926 19:34:00.669941 15161 raft_consensus.cc:2258] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:00.670135 15162 tablet_replica.cc:330] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: stopping tablet replica
I20260926 19:34:00.670300 15162 raft_consensus.cc:2229] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:34:00.670562 15162 raft_consensus.cc:2258] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:00.671520 15162 ts_tablet_manager.cc:1905] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:00.672292 14581 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:34:00.673033 14581 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:34:00.676399 15162 ts_tablet_manager.cc:1918] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:34:00.676599 15162 log.cc:1196] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/9f31fc6ba31c4d68ac67ce28c9b4e990
I20260926 19:34:00.677390 15162 ts_tablet_manager.cc:1939] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting consensus metadata
I20260926 19:34:00.677874 15163 ts_tablet_manager.cc:1905] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:00.679075 15161 ts_tablet_manager.cc:1905] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:00.681051 14857 tablet_service.cc:1467] Processing CreateTablet for tablet 7023074c656046d79bc1fb9d4b7c93e8 (DEFAULT_TABLE table=test-table [id=e511dab5b03e40a983fd65a8d8f23568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:00.681500 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7023074c656046d79bc1fb9d4b7c93e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.685052 15163 ts_tablet_manager.cc:1918] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:34:00.685348 15163 log.cc:1196] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/9f31fc6ba31c4d68ac67ce28c9b4e990
I20260926 19:34:00.685549 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 7023074c656046d79bc1fb9d4b7c93e8 (DEFAULT_TABLE table=test-table [id=e511dab5b03e40a983fd65a8d8f23568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:00.685720 15161 ts_tablet_manager.cc:1918] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:34:00.685875 15161 log.cc:1196] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/9f31fc6ba31c4d68ac67ce28c9b4e990
I20260926 19:34:00.686021 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7023074c656046d79bc1fb9d4b7c93e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.686050 15163 ts_tablet_manager.cc:1939] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting consensus metadata
I20260926 19:34:00.686300 15161 ts_tablet_manager.cc:1939] T 9f31fc6ba31c4d68ac67ce28c9b4e990 P 9d723907ea6e4bcba154190572fdca9e: Deleting consensus metadata
I20260926 19:34:00.687271 14567 catalog_manager.cc:4841] TS 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619): tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 (table test-table [id=5f6f3a9f5c724c57940fdb3c28db29cf]) successfully deleted
W20260926 19:34:00.688452 14567 catalog_manager.cc:4808] TS 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619): delete failed for tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 because the tablet was not found. No further retry: Not found: Tablet not found: 9f31fc6ba31c4d68ac67ce28c9b4e990
I20260926 19:34:00.689024 14568 catalog_manager.cc:4841] TS 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843): tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 (table test-table [id=5f6f3a9f5c724c57940fdb3c28db29cf]) successfully deleted
I20260926 19:34:00.689198 14569 catalog_manager.cc:4841] TS 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639): tablet 9f31fc6ba31c4d68ac67ce28c9b4e990 (table test-table [id=5f6f3a9f5c724c57940fdb3c28db29cf]) successfully deleted
I20260926 19:34:00.689847 14719 tablet_service.cc:1467] Processing CreateTablet for tablet 7023074c656046d79bc1fb9d4b7c93e8 (DEFAULT_TABLE table=test-table [id=e511dab5b03e40a983fd65a8d8f23568]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:00.690286 14719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7023074c656046d79bc1fb9d4b7c93e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:00.693156 15092 tablet_bootstrap.cc:492] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: Bootstrap starting.
I20260926 19:34:00.694767 15092 tablet_bootstrap.cc:654] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.696465 15095 tablet_bootstrap.cc:492] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: Bootstrap starting.
I20260926 19:34:00.698091 15095 tablet_bootstrap.cc:654] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.700176 15092 tablet_bootstrap.cc:492] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: No bootstrap required, opened a new log
I20260926 19:34:00.700347 15092 ts_tablet_manager.cc:1397] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.001s
I20260926 19:34:00.701236 15092 raft_consensus.cc:348] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.701489 15092 raft_consensus.cc:374] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.701574 15092 raft_consensus.cc:729] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d723907ea6e4bcba154190572fdca9e, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.701853 15092 consensus_queue.cc:260] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.702476 15092 ts_tablet_manager.cc:1428] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 19:34:00.702850 15093 tablet_bootstrap.cc:492] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: Bootstrap starting.
I20260926 19:34:00.704288 15093 tablet_bootstrap.cc:654] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:00.723476 15093 tablet_bootstrap.cc:492] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: No bootstrap required, opened a new log
I20260926 19:34:00.723668 15093 ts_tablet_manager.cc:1397] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.007s
I20260926 19:34:00.724020 15095 tablet_bootstrap.cc:492] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: No bootstrap required, opened a new log
I20260926 19:34:00.724210 15095 ts_tablet_manager.cc:1397] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.015s
I20260926 19:34:00.724498 15093 raft_consensus.cc:348] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.724751 15093 raft_consensus.cc:374] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.724860 15093 raft_consensus.cc:729] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e69b2bc8c814323b9e4f0ee56278a37, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.725004 15095 raft_consensus.cc:348] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.725248 15095 raft_consensus.cc:374] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:00.725127 15093 consensus_queue.cc:260] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.725323 15095 raft_consensus.cc:729] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd2fabb62d84e9e9fafd9061dee4252, State: Initialized, Role: FOLLOWER
I20260926 19:34:00.725526 15095 consensus_queue.cc:260] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.727321 15093 ts_tablet_manager.cc:1428] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260926 19:34:00.729054 15095 ts_tablet_manager.cc:1428] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 19:34:00.894826 15098 raft_consensus.cc:482] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:34:00.895048 15098 raft_consensus.cc:504] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.896454 15098 leader_election.cc:288] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:34:00.902411 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.896852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:48949 (local address 127.6.215.195:34843)
0926 19:34:00.897828 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.897834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.897856 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:34:00.897924 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.897927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.897988 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.898099 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:00.898500 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.899181 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.900832 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.901522 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.901688 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.901775 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.901997 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.902124 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":641,"threads_started":1}
I20260926 19:34:00.902474 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.896736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34843 (local address 127.6.215.194:48949)
0926 19:34:00.897519 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.897539 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.897685 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.898116 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.898121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.898148 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:00.898365 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.898373 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.899317 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.899322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.900647 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.900663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.901675 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.901681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.901838 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.901874 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.901975 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.902105 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":434,"threads_started":1}
I20260926 19:34:00.902951 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.896794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34639 (local address 127.6.215.194:51045)
0926 19:34:00.897982 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.897996 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.898122 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.898436 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.898441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.898461 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:00.898771 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.898778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.899701 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.899709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.901125 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.901135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.902213 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.902220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.902371 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.902395 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.902501 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.902637 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"thread_start_us":945,"threads_started":1}
I20260926 19:34:00.903059 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7023074c656046d79bc1fb9d4b7c93e8" candidate_uuid: "9d723907ea6e4bcba154190572fdca9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" is_pre_election: true
I20260926 19:34:00.903228 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.896852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:51045 (local address 127.6.215.193:34639)
0926 19:34:00.897053 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.897062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.898088 (+ 1026us) server_negotiation.cc:407] Connection header received
0926 19:34:00.898259 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.898263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.898331 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.898430 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:00.898929 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.899572 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.901264 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.902004 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.902184 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.902256 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.902827 (+ 571us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.902973 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:34:00.903345 15017 raft_consensus.cc:2454] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d723907ea6e4bcba154190572fdca9e in term 0.
I20260926 19:34:00.903810 14739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7023074c656046d79bc1fb9d4b7c93e8" candidate_uuid: "9d723907ea6e4bcba154190572fdca9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" is_pre_election: true
I20260926 19:34:00.903887 14811 leader_election.cc:302] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e69b2bc8c814323b9e4f0ee56278a37, 9d723907ea6e4bcba154190572fdca9e; no voters:
I20260926 19:34:00.904059 14739 raft_consensus.cc:2454] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d723907ea6e4bcba154190572fdca9e in term 0.
I20260926 19:34:00.904290 15098 raft_consensus.cc:2781] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:34:00.904422 15098 raft_consensus.cc:482] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:34:00.904523 15098 raft_consensus.cc:3037] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.907351 15098 raft_consensus.cc:504] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.907910 15098 leader_election.cc:288] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [CANDIDATE]: Term 1 election: Requested vote from peers 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639), 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843)
I20260926 19:34:00.908241 14739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7023074c656046d79bc1fb9d4b7c93e8" candidate_uuid: "9d723907ea6e4bcba154190572fdca9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dd2fabb62d84e9e9fafd9061dee4252"
I20260926 19:34:00.908432 14739 raft_consensus.cc:3037] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.908401 15017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7023074c656046d79bc1fb9d4b7c93e8" candidate_uuid: "9d723907ea6e4bcba154190572fdca9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e69b2bc8c814323b9e4f0ee56278a37"
I20260926 19:34:00.908592 15017 raft_consensus.cc:3037] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:00.910597 14739 raft_consensus.cc:2454] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d723907ea6e4bcba154190572fdca9e in term 1.
I20260926 19:34:00.910742 15017 raft_consensus.cc:2454] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d723907ea6e4bcba154190572fdca9e in term 1.
I20260926 19:34:00.910993 14812 leader_election.cc:302] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dd2fabb62d84e9e9fafd9061dee4252, 9d723907ea6e4bcba154190572fdca9e; no voters:
I20260926 19:34:00.911254 15098 raft_consensus.cc:2781] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:00.911705 15098 raft_consensus.cc:686] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 1 LEADER]: Becoming Leader. State: Replica: 9d723907ea6e4bcba154190572fdca9e, State: Running, Role: LEADER
I20260926 19:34:00.912068 15098 consensus_queue.cc:237] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } }
I20260926 19:34:00.915643 14582 catalog_manager.cc:5495] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d723907ea6e4bcba154190572fdca9e (127.6.215.194). New cstate: current_term: 1 leader_uuid: "9d723907ea6e4bcba154190572fdca9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d723907ea6e4bcba154190572fdca9e" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 } health_report { overall_health: UNKNOWN } } }
I20260926 19:34:00.957672 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.952370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.6.215.194:37619)
0926 19:34:00.952929 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.952933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.952951 (+ 18us) server_negotiation.cc:407] Connection header received
0926 19:34:00.953024 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.953027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.953094 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.953232 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:00.953641 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.954362 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.955611 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.956010 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.956105 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.956518 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.957108 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.957213 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.957351 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.957413 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":321,"threads_started":1}
I20260926 19:34:00.972039 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.966902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:36403 (local address 127.6.215.195:34843)
0926 19:34:00.967074 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.967079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:00.967688 (+ 609us) server_negotiation.cc:407] Connection header received
0926 19:34:00.967798 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:00.967802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:00.967867 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:00.967958 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:00.968427 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.969261 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.970519 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.970946 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.971049 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.971414 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.971515 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.971606 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:00.971781 (+ 175us) server_negotiation.cc:299] Negotiation successful
0926 19:34:00.971840 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 19:34:00.972075 15197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.966862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34843 (local address 127.6.215.194:36403)
0926 19:34:00.967571 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:00.967586 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:00.967697 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:00.967969 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:00.967972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:00.967980 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:34:00.968250 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.968257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.969374 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.969377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.970376 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:00.970386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:00.971091 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:00.971095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:00.971203 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:00.971248 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:34:00.971612 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:34:00.971627 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 19:34:00.971697 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 19:34:00.971788 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":374,"threads_started":1}
I20260926 19:34:00.980448 14739 raft_consensus.cc:1261] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Refusing update from remote peer 9d723907ea6e4bcba154190572fdca9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.980446 15017 raft_consensus.cc:1261] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Refusing update from remote peer 9d723907ea6e4bcba154190572fdca9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:00.981171 15189 consensus_queue.cc:1035] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dd2fabb62d84e9e9fafd9061dee4252" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:00.981478 15098 consensus_queue.cc:1035] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e69b2bc8c814323b9e4f0ee56278a37" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:01.005661 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.999371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:44913 (local address 127.6.215.194:37619)
0926 19:34:00.999557 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:00.999566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:01.001215 (+ 1649us) server_negotiation.cc:407] Connection header received
0926 19:34:01.001332 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:01.001335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:01.001391 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:01.001468 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:01.001842 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.002673 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.004303 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.004699 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.004841 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:01.005132 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:01.005218 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:01.005284 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:01.005391 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 19:34:01.005452 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 19:34:01.005815 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:00.999363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37619 (local address 127.6.215.195:44913)
0926 19:34:01.001128 (+ 1765us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:01.001144 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:01.001249 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:01.001496 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:01.001499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:01.001508 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 19:34:01.001672 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:01.001678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.002819 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.002826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:01.004177 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:01.004189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.004871 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.004876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:01.004983 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:01.005016 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 19:34:01.005296 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 19:34:01.005303 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 19:34:01.005361 (+ 58us) client_negotiation.cc:240] Negotiation successful
0926 19:34:01.005451 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1657,"thread_start_us":1546,"threads_started":1}
I20260926 19:34:01.042889 14581 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:01.043164 14581 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:01.046406 14581 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c1447da7f0040fda9bab90ce1a972fa: Sending DeleteTablet for 3 replicas of tablet 7023074c656046d79bc1fb9d4b7c93e8
I20260926 19:34:01.047319 14719 tablet_service.cc:1514] Processing DeleteTablet for tablet 7023074c656046d79bc1fb9d4b7c93e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:01 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45778
I20260926 19:34:01.047631 15162 tablet_replica.cc:330] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: stopping tablet replica
I20260926 19:34:01.047811 15162 raft_consensus.cc:2229] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:01.047827 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 7023074c656046d79bc1fb9d4b7c93e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:01 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:44692
I20260926 19:34:01.048044 15162 raft_consensus.cc:2258] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:01.048131 15163 tablet_replica.cc:330] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: stopping tablet replica
I20260926 19:34:01.048167 14857 tablet_service.cc:1514] Processing DeleteTablet for tablet 7023074c656046d79bc1fb9d4b7c93e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:01 UTC) from {username='oryx', principal='oryx/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:49734
I20260926 19:34:01.048300 15163 raft_consensus.cc:2229] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:01.048444 15161 tablet_replica.cc:330] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: stopping tablet replica
I20260926 19:34:01.048497 15163 raft_consensus.cc:2258] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:01.048640 15161 raft_consensus.cc:2229] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:34:01.049109 15161 raft_consensus.cc:2258] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:01.049270 15162 ts_tablet_manager.cc:1905] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:01.050210 15161 ts_tablet_manager.cc:1905] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:01.050402 15163 ts_tablet_manager.cc:1905] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:01.053979 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 14653
I20260926 19:34:01.057472 15163 ts_tablet_manager.cc:1918] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:34:01.057622 15163 log.cc:1196] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/7023074c656046d79bc1fb9d4b7c93e8
I20260926 19:34:01.058168 15163 ts_tablet_manager.cc:1939] T 7023074c656046d79bc1fb9d4b7c93e8 P 7e69b2bc8c814323b9e4f0ee56278a37: Deleting consensus metadata
I20260926 19:34:01.059604 14568 catalog_manager.cc:4841] TS 7e69b2bc8c814323b9e4f0ee56278a37 (127.6.215.195:34843): tablet 7023074c656046d79bc1fb9d4b7c93e8 (table test-table [id=e511dab5b03e40a983fd65a8d8f23568]) successfully deleted
I20260926 19:34:01.060142 15162 ts_tablet_manager.cc:1918] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:34:01.060271 15162 log.cc:1196] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/7023074c656046d79bc1fb9d4b7c93e8
I20260926 19:34:01.060719 15162 ts_tablet_manager.cc:1939] T 7023074c656046d79bc1fb9d4b7c93e8 P 4dd2fabb62d84e9e9fafd9061dee4252: Deleting consensus metadata
I20260926 19:34:01.061272 15161 ts_tablet_manager.cc:1918] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 19:34:01.061460 15161 log.cc:1196] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/7023074c656046d79bc1fb9d4b7c93e8
I20260926 19:34:01.061813 14569 catalog_manager.cc:4841] TS 4dd2fabb62d84e9e9fafd9061dee4252 (127.6.215.193:34639): tablet 7023074c656046d79bc1fb9d4b7c93e8 (table test-table [id=e511dab5b03e40a983fd65a8d8f23568]) successfully deleted
I20260926 19:34:01.062040 15161 ts_tablet_manager.cc:1939] T 7023074c656046d79bc1fb9d4b7c93e8 P 9d723907ea6e4bcba154190572fdca9e: Deleting consensus metadata
I20260926 19:34:01.063198 14567 catalog_manager.cc:4841] TS 9d723907ea6e4bcba154190572fdca9e (127.6.215.194:37619): tablet 7023074c656046d79bc1fb9d4b7c93e8 (table test-table [id=e511dab5b03e40a983fd65a8d8f23568]) successfully deleted
Sep 26 19:34:01 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.193@KRBTEST.COM
I20260926 19:34:01.077255 15133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:01.057149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.6.215.193:34639)
0926 19:34:01.057325 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:01.057331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:01.060548 (+ 3217us) server_negotiation.cc:407] Connection header received
0926 19:34:01.060823 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:01.060830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:01.060914 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:01.061002 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:01.062779 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.063883 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.065668 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.066058 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.066198 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:01.070917 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:01.070933 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:01.070950 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:01.070991 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:01.074197 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:01.075055 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:01.075062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:01.075064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:01.075267 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:01.075660 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:01.075663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:01.075665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:01.076154 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:01.076442 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:01.076736 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 19:34:01.076955 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:34:01.078274 14784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:01.205255 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 14653
I20260926 19:34:01.229033 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 14793
Sep 26 19:34:01 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.194@KRBTEST.COM
I20260926 19:34:01.248711 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:01.229872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49760 (local address 127.6.215.194:37619)
0926 19:34:01.230069 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:01.230075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:01.230160 (+ 85us) server_negotiation.cc:407] Connection header received
0926 19:34:01.230370 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:01.230376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:01.230451 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:01.230553 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:01.232440 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.233639 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.235359 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.236027 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.236153 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:01.241441 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:01.241453 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:01.241469 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:01.241515 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:01.245480 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:01.246304 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:01.246314 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:01.246318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:01.246569 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:01.247019 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:01.247025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:01.247029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:01.247539 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:01.247858 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:01.248149 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 19:34:01.248362 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 19:34:01.249581 14923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:01.372036 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 14793
I20260926 19:34:01.392099 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 14932
Sep 26 19:34:01 dist-test-slave-phbh krb5kdc[14537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.215.195@KRBTEST.COM
I20260926 19:34:01.407842 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:01.392851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.6.215.195:34843)
0926 19:34:01.393024 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:01.393028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:01.393195 (+ 167us) server_negotiation.cc:407] Connection header received
0926 19:34:01.393354 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:01.393358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:01.393421 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:01.393500 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:01.394811 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.395528 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.396864 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:01.397258 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:01.397364 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:01.401867 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:01.401881 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:01.401915 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:01.401954 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:01.404635 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:01.405712 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:01.405723 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:01.405732 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:01.405875 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:01.406249 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:01.406253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:01.406255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:01.406593 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:01.406882 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:01.407229 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 19:34:01.407436 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:34:01.408738 15061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:01.537403 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 14932
I20260926 19:34:01.558972 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 14550
I20260926 19:34:01.560107 14622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:01.673257 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 14550
2026-09-26T19:34:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6865 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:34:01 dist-test-slave-phbh krb5kdc[15213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:34:01 dist-test-slave-phbh krb5kdc[15213](info): set up 2 sockets
Sep 26 19:34:01 dist-test-slave-phbh krb5kdc[15213](info): commencing operation
krb5kdc: starting...
W20260926 19:34:02.777892 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:34:02 dist-test-slave-phbh krb5kdc[15213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:34:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:34:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:34:03.000038 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:38267
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:03.326756 15226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:03.327049 15226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:03.327214 15226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:03.334416 15226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:34:03.334527 15226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:03.334625 15226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:03.334787 15226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:34:03.334872 15226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:34:03.344619 15226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:38267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:03.345954 15226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:03.347282 15226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:03.358887 15231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:03.358912 15234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:03.358846 15232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:03.359917 15226 server_base.cc:1034] running on GCE node
I20260926 19:34:03.360661 15226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:03.362066 15226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:03.363276 15226 hybrid_clock.cc:648] HybridClock initialized: now 1790451243363222 us; error 36 us; skew 500 ppm
I20260926 19:34:03.363620 15226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T19:34:03Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1763 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:34:03 dist-test-slave-phbh krb5kdc[15240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:34:03 dist-test-slave-phbh krb5kdc[15240](info): set up 2 sockets
Sep 26 19:34:03 dist-test-slave-phbh krb5kdc[15240](info): commencing operation
krb5kdc: starting...
W20260926 19:34:04.545768 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:34:04 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:34:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:34:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:34:04.763962 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40257
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:40257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:05.084064 15253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:05.084376 15253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:05.084465 15253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:05.092029 15253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:34:05.092159 15253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:05.092204 15253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:05.092260 15253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:34:05.092304 15253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:34:05.102389 15253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:40257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:05.103745 15253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:05.105038 15253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:05.116565 15258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:05.116603 15259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:05.116637 15261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:05.117563 15253 server_base.cc:1034] running on GCE node
I20260926 19:34:05.118299 15253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:05.119673 15253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:05.120900 15253 hybrid_clock.cc:648] HybridClock initialized: now 1790451245120847 us; error 61 us; skew 500 ppm
I20260926 19:34:05.121223 15253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:05 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451245, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:05.129485 15253 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:34:05.130918 15253 webserver.cc:460] Webserver started at http://127.6.215.254:34631/ using document root <none> and password file <none>
I20260926 19:34:05.131618 15253 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:05.131753 15253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:05.132056 15253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:05.135010 15253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "2679468b5c264c839c2841d16ad8183f"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "073dd422b5cbe9af71b92cc32dc54f27"
server_key_iv: "4d8003cb187ce7128158b335252ae3c5"
server_key_version: "encryptionkey@0"
I20260926 19:34:05.135763 15253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "2679468b5c264c839c2841d16ad8183f"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "073dd422b5cbe9af71b92cc32dc54f27"
server_key_iv: "4d8003cb187ce7128158b335252ae3c5"
server_key_version: "encryptionkey@0"
I20260926 19:34:05.140625 15253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260926 19:34:05.143810 15268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:05.145103 15253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:34:05.145282 15253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2679468b5c264c839c2841d16ad8183f"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "073dd422b5cbe9af71b92cc32dc54f27"
server_key_iv: "4d8003cb187ce7128158b335252ae3c5"
server_key_version: "encryptionkey@0"
I20260926 19:34:05.145475 15253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:05.168996 15253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:05.173040 15253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:05.173322 15253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:05.190909 15253 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:40257
I20260926 19:34:05.190928 15319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:40257 every 8 connection(s)
I20260926 19:34:05.192214 15253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:34:05.194528 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 15253
I20260926 19:34:05.194722 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:34:05.195369 7007 external_mini_cluster.cc:1428] Setting key 2d17fe089fe1c3855b9306e907ef650d
I20260926 19:34:05.195983 15320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:05.210565 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Bootstrap starting.
Sep 26 19:34:05 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:05.214519 15320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:05.215878 15320 log.cc:824] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:05.218859 15320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: No bootstrap required, opened a new log
I20260926 19:34:05.219971 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:05.197439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.6.215.254:40257)
0926 19:34:05.198180 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:05.198192 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:05.198228 (+ 36us) server_negotiation.cc:407] Connection header received
0926 19:34:05.200478 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:05.200497 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:05.200967 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:05.201288 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:05.203018 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:05.204275 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:05.206007 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:05.206679 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:05.207667 (+ 988us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:05.211887 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:05.211903 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:05.211922 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:05.211978 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:05.215946 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:05.216645 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:05.216652 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:05.216657 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:05.216886 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:05.217268 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:05.217273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:05.217277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:05.217782 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:05.218073 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:05.218552 (+ 479us) server_negotiation.cc:299] Negotiation successful
0926 19:34:05.218990 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":346,"threads_started":1}
I20260926 19:34:05.226491 15320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:05.226833 15320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:05.226897 15320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2679468b5c264c839c2841d16ad8183f, State: Initialized, Role: FOLLOWER
I20260926 19:34:05.227541 15320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:05.227810 15320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:34:05.227993 15320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:34:05.228169 15320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:05.230291 15320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:05.230726 15320 leader_election.cc:302] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2679468b5c264c839c2841d16ad8183f; no voters:
I20260926 19:34:05.231308 15320 leader_election.cc:288] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:34:05.231446 15325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:05.232121 15325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 LEADER]: Becoming Leader. State: Replica: 2679468b5c264c839c2841d16ad8183f, State: Running, Role: LEADER
I20260926 19:34:05.232651 15325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:05.233383 15320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:34:05.235472 15326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2679468b5c264c839c2841d16ad8183f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } } }
I20260926 19:34:05.235845 15326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:05.237984 15330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:34:05.237932 15327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2679468b5c264c839c2841d16ad8183f. Latest consensus state: current_term: 1 leader_uuid: "2679468b5c264c839c2841d16ad8183f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } } }
I20260926 19:34:05.238242 15327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:05.243009 15330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:34:05.251099 15330 catalog_manager.cc:1347] Generated new cluster ID: 2a586095111c45b5835f775363ccdfd4
I20260926 19:34:05.251216 15330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:34:05.268841 15330 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:34:05.270373 15330 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:34:05.279383 15330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Generated new TSK 0
I20260926 19:34:05.280205 15330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:34:05.348443 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40257
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:05.686975 15346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:05.687299 15346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:05.687458 15346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:05.694602 15346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:05.694756 15346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:05.694928 15346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:34:05.704540 15346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:05.705825 15346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:05.707065 15346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:05.719745 15352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:05.720885 15351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:05.720959 15354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:05.720959 15346 server_base.cc:1034] running on GCE node
I20260926 19:34:05.721637 15346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:05.722456 15346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:05.723650 15346 hybrid_clock.cc:648] HybridClock initialized: now 1790451245723607 us; error 33 us; skew 500 ppm
I20260926 19:34:05.723910 15346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:05 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451245, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:05.726853 15346 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:34:05.728268 15346 webserver.cc:460] Webserver started at http://127.6.215.193:44879/ using document root <none> and password file <none>
I20260926 19:34:05.728965 15346 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:05.729087 15346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:05.729391 15346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:05.732213 15346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "437dc7e438254da1bdf82fc851f97e16"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "93896180af0a4d350d87b6963151024f"
server_key_iv: "20c18898e36b2a37d5b78760b5fba674"
server_key_version: "encryptionkey@0"
I20260926 19:34:05.733031 15346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "437dc7e438254da1bdf82fc851f97e16"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "93896180af0a4d350d87b6963151024f"
server_key_iv: "20c18898e36b2a37d5b78760b5fba674"
server_key_version: "encryptionkey@0"
I20260926 19:34:05.738125 15346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 19:34:05.741456 15361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:05.742774 15346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 19:34:05.742956 15346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "437dc7e438254da1bdf82fc851f97e16"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "93896180af0a4d350d87b6963151024f"
server_key_iv: "20c18898e36b2a37d5b78760b5fba674"
server_key_version: "encryptionkey@0"
I20260926 19:34:05.743139 15346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:05.772029 15346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:05.776175 15346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:05.776563 15346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:05.777626 15346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:05.779066 15346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:05.779165 15346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:05.779304 15346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:05.779372 15346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:05.811299 15346 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:46181
I20260926 19:34:05.811344 15473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:46181 every 8 connection(s)
I20260926 19:34:05.812497 15346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:34:05.813083 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 15346
I20260926 19:34:05.813294 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:34:05.814152 7007 external_mini_cluster.cc:1428] Setting key b9a34baa8520671f27ad9cbc1b7b2865
Sep 26 19:34:05 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451245, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:05.833034 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:05.815236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:50661 (local address 127.6.215.254:40257)
0926 19:34:05.815675 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:05.815679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:05.816744 (+ 1065us) server_negotiation.cc:407] Connection header received
0926 19:34:05.818541 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:05.818546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:05.818631 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:05.818749 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:05.820262 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:05.821030 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:05.823080 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:05.823479 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:05.823584 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:05.827621 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:05.827626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:05.827629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:05.827658 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:05.830099 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:05.830795 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:05.830802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:05.830806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:05.830928 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:05.831333 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:05.831339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:05.831342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:05.831591 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:05.831753 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:05.832416 (+ 663us) server_negotiation.cc:299] Negotiation successful
0926 19:34:05.832630 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":232,"threads_started":1}
I20260926 19:34:05.834116 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:05.815937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:50661)
0926 19:34:05.816605 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:05.816642 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:05.818301 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:05.818970 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:05.818978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:05.819576 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:05.820108 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:05.820124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:05.821180 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:05.821185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:05.822930 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:05.822947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:05.823700 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:05.823709 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:05.824031 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:05.824745 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:05.824794 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:05.827410 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:05.830247 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:05.830255 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:05.830272 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:05.830678 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:05.831036 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:05.831041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:05.831044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:05.831224 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:05.831728 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:05.831736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:05.832070 (+ 334us) client_negotiation.cc:769] Sending connection context
0926 19:34:05.832326 (+ 256us) client_negotiation.cc:240] Negotiation successful
0926 19:34:05.832886 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":284,"threads_started":1}
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:34:05.835655 15474 heartbeater.cc:347] Connected to a master server at 127.6.215.254:40257
I20260926 19:34:05.836012 15474 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:05.836699 15474 heartbeater.cc:511] Master 127.6.215.254:40257 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
I20260926 19:34:05.839152 15285 ts_manager.cc:194] Registered new tserver with Master: 437dc7e438254da1bdf82fc851f97e16 (127.6.215.193:46181)
I20260926 19:34:05.841287 15285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:50661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:34:05.864060 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40257
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:06.181103 15480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:06.181437 15480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:06.181563 15480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:06.188859 15480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:06.188969 15480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:06.189078 15480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:34:06.203652 15480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:06.205282 15480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:06.206832 15480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:06.221045 15486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:06.221025 15488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:06.221237 15485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:06.221824 15480 server_base.cc:1034] running on GCE node
I20260926 19:34:06.222390 15480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:06.223274 15480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:06.224468 15480 hybrid_clock.cc:648] HybridClock initialized: now 1790451246224432 us; error 25 us; skew 500 ppm
I20260926 19:34:06.224745 15480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:06 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451246, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:06.227757 15480 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:34:06.229225 15480 webserver.cc:460] Webserver started at http://127.6.215.194:41945/ using document root <none> and password file <none>
I20260926 19:34:06.229898 15480 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:06.230020 15480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:06.230322 15480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:06.233294 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "3cb6bbf1570a43a3ad9e4ecae8a48eb1"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "c46737d16c7c3eff9e6c1461696c2bcf"
server_key_iv: "25f0322dfbada8c4c36b945d71f7590c"
server_key_version: "encryptionkey@0"
I20260926 19:34:06.234115 15480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "3cb6bbf1570a43a3ad9e4ecae8a48eb1"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "c46737d16c7c3eff9e6c1461696c2bcf"
server_key_iv: "25f0322dfbada8c4c36b945d71f7590c"
server_key_version: "encryptionkey@0"
I20260926 19:34:06.239274 15480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:34:06.242619 15495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:06.243907 15480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:34:06.244074 15480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "3cb6bbf1570a43a3ad9e4ecae8a48eb1"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "c46737d16c7c3eff9e6c1461696c2bcf"
server_key_iv: "25f0322dfbada8c4c36b945d71f7590c"
server_key_version: "encryptionkey@0"
I20260926 19:34:06.244275 15480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:06.291795 15480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:06.296020 15480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:06.296312 15480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:06.297313 15480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:06.298612 15480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:06.298753 15480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:06.298857 15480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:06.298909 15480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:06.332135 15480 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:42835
I20260926 19:34:06.332191 15607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:42835 every 8 connection(s)
I20260926 19:34:06.333545 15480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:34:06.335053 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 15480
I20260926 19:34:06.335238 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:34:06.335806 7007 external_mini_cluster.cc:1428] Setting key ee4d1dfb465614d5b4463e4b434601e5
Sep 26 19:34:06 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451246, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:06.359510 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:06.336459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:33051 (local address 127.6.215.254:40257)
0926 19:34:06.336933 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:06.336938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:06.340861 (+ 3923us) server_negotiation.cc:407] Connection header received
0926 19:34:06.341275 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:06.341279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:06.341341 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:06.341455 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:06.343569 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.344277 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.346735 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.347623 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.347872 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:06.351590 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:06.351601 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:06.351610 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:06.351668 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:06.355763 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:06.356563 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:06.356572 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:06.356579 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:06.356813 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:06.357243 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:06.357252 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:06.357259 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:06.357764 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:06.357942 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:06.358703 (+ 761us) server_negotiation.cc:299] Negotiation successful
0926 19:34:06.358978 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":249,"threads_started":1}
I20260926 19:34:06.360080 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:06.337138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:33051)
0926 19:34:06.338076 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:06.338128 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:06.340960 (+ 2832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:06.341752 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:06.341766 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:06.342540 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:06.343374 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:06.343402 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.344450 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.344459 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:06.346501 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:06.346515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.347985 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.347993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:06.348226 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:06.349130 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:06.349164 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:06.351327 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:06.355946 (+ 4619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:06.355953 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:06.355971 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:06.356411 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:06.356924 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:06.356928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:06.356930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:06.357115 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:06.357901 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:06.357908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:06.358246 (+ 338us) client_negotiation.cc:769] Sending connection context
0926 19:34:06.358487 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 19:34:06.358909 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
I20260926 19:34:06.361609 15608 heartbeater.cc:347] Connected to a master server at 127.6.215.254:40257
I20260926 19:34:06.361994 15608 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:34:06.362901 15608 heartbeater.cc:511] Master 127.6.215.254:40257 requested a full tablet report, sending...
I20260926 19:34:06.364445 15285 ts_manager.cc:194] Registered new tserver with Master: 3cb6bbf1570a43a3ad9e4ecae8a48eb1 (127.6.215.194:42835)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
I20260926 19:34:06.365639 15285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:33051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:34:06.391494 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40257
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:06.726164 15614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:06.726454 15614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:06.726545 15614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:06.733933 15614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:06.734058 15614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:06.734174 15614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:34:06.744386 15614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:06.745738 15614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:06.747035 15614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:06.759860 15620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:06.759826 15619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:06.759883 15622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:06.760756 15614 server_base.cc:1034] running on GCE node
I20260926 19:34:06.761379 15614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:06.762323 15614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:06.763542 15614 hybrid_clock.cc:648] HybridClock initialized: now 1790451246763493 us; error 44 us; skew 500 ppm
I20260926 19:34:06.763887 15614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:06 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451246, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:06.766940 15614 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:34:06.768383 15614 webserver.cc:460] Webserver started at http://127.6.215.195:37003/ using document root <none> and password file <none>
I20260926 19:34:06.769119 15614 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:06.769251 15614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:06.769541 15614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:06.772301 15614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "9e38e1d97a6342078fed53662dab8053"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "ccc1a4d12215f71ce98fd4ba92b3b02f"
server_key_iv: "cfe0cf2bc1af4d2071f02f12f89e00aa"
server_key_version: "encryptionkey@0"
I20260926 19:34:06.773126 15614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "9e38e1d97a6342078fed53662dab8053"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "ccc1a4d12215f71ce98fd4ba92b3b02f"
server_key_iv: "cfe0cf2bc1af4d2071f02f12f89e00aa"
server_key_version: "encryptionkey@0"
I20260926 19:34:06.778102 15614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 19:34:06.781564 15629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:06.782956 15614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:34:06.783126 15614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "9e38e1d97a6342078fed53662dab8053"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "ccc1a4d12215f71ce98fd4ba92b3b02f"
server_key_iv: "cfe0cf2bc1af4d2071f02f12f89e00aa"
server_key_version: "encryptionkey@0"
I20260926 19:34:06.783326 15614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:06.797016 15614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:06.800911 15614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:06.801195 15614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:06.802083 15614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:06.803335 15614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:06.803442 15614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:06.803579 15614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:06.803647 15614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:06.836683 15614 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:46617
I20260926 19:34:06.836727 15741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:46617 every 8 connection(s)
I20260926 19:34:06.837975 15614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:34:06.843196 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 15614
I20260926 19:34:06.843371 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:34:06.843932 7007 external_mini_cluster.cc:1428] Setting key e6eb8efb083fdd36c3a5fe90b8999a05
I20260926 19:34:06.844224 15474 heartbeater.cc:503] Master 127.6.215.254:40257 was elected leader, sending a full tablet report...
Sep 26 19:34:06 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451246, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:06.858290 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:06.840397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:35793 (local address 127.6.215.254:40257)
0926 19:34:06.840597 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:06.840603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:06.841835 (+ 1232us) server_negotiation.cc:407] Connection header received
0926 19:34:06.843973 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:06.843994 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:06.844094 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:06.844207 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:06.846198 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.847292 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.849051 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.849501 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.849594 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:06.852859 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:06.852864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:06.852868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:06.852899 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:06.855471 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:06.856086 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:06.856089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:06.856091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:06.856218 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:06.856557 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:06.856560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:06.856562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:06.856899 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:06.857045 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:06.857721 (+ 676us) server_negotiation.cc:299] Negotiation successful
0926 19:34:06.857932 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 19:34:06.859309 15744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:06.841032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:35793)
0926 19:34:06.841684 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:06.841720 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:06.843678 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:06.844472 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:06.844483 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:06.845292 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:06.846059 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:06.846077 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.847453 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.847457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:06.848945 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:06.848955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:06.849675 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:06.849680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:06.849895 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:06.850501 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:06.850522 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:06.852586 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:06.855578 (+ 2992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:06.855584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:06.855596 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:06.855987 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:06.856302 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:06.856305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:06.856307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:06.856466 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:06.857021 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:06.857026 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:06.857309 (+ 283us) client_negotiation.cc:769] Sending connection context
0926 19:34:06.857566 (+ 257us) client_negotiation.cc:240] Negotiation successful
0926 19:34:06.858034 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":245,"threads_started":1}
I20260926 19:34:06.860688 15742 heartbeater.cc:347] Connected to a master server at 127.6.215.254:40257
I20260926 19:34:06.861017 15742 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:06.861642 15742 heartbeater.cc:511] Master 127.6.215.254:40257 requested a full tablet report, sending...
I20260926 19:34:06.862927 15285 ts_manager.cc:194] Registered new tserver with Master: 9e38e1d97a6342078fed53662dab8053 (127.6.215.195:46617)
I20260926 19:34:06.863643 15285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:35793
I20260926 19:34:06.871295 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab.
Entry for principal oryx/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab.
WARNING: no policy specified for oryx/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab.
Entry for principal oryx/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab.
WARNING: no policy specified for oryx/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab.
Entry for principal oryx/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab.
I20260926 19:34:07.004657 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15346
Sep 26 19:34:07 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:34:07.021231 15752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:07.005912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.6.215.193:46181)
0926 19:34:07.006616 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:07.006623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:07.006650 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:34:07.006710 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:07.006716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:07.006939 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:07.007081 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:07.008421 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:07.009723 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:07.010976 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:07.011606 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:07.011749 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:07.015697 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:07.015709 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:07.015725 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:07.015763 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:07.018704 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:07.019308 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:07.019315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:07.019317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:07.019443 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:07.019733 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:07.019736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:07.019738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:07.020101 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:07.020317 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:07.020664 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 19:34:07.020938 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":393,"threads_started":1}
I20260926 19:34:07.022531 15469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:07.107733 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 15346
I20260926 19:34:07.123986 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15480
Sep 26 19:34:07 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:34:07.139353 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:07.124935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.6.215.194:42835)
0926 19:34:07.125464 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:07.125471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:07.125496 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:34:07.125553 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:07.125560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:07.125768 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:07.125903 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:07.127029 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:07.128040 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:07.129330 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:07.129872 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:07.129969 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:07.134100 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:07.134111 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:07.134126 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:07.134159 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:07.136758 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:07.137488 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:07.137495 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:07.137498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:07.137610 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:07.137919 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:07.137922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:07.137924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:07.138293 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:07.138546 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:07.138808 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 19:34:07.139005 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":255,"threads_started":1}
I20260926 19:34:07.140627 15603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:07.222733 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 15480
I20260926 19:34:07.238574 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15614
Sep 26 19:34:07 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:34:07.254540 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:07.239551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.6.215.195:46617)
0926 19:34:07.240114 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:07.240124 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:07.240166 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:34:07.240226 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:07.240232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:07.240448 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:07.240615 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:07.241919 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:07.242945 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:07.244202 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:07.244761 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:07.244919 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:07.249270 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:07.249283 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:07.249298 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:07.249338 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:07.251974 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:07.252588 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:07.252595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:07.252598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:07.252716 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:07.253139 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:07.253142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:07.253144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:07.253485 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:07.253720 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:07.254048 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 19:34:07.254243 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":261,"threads_started":1}
I20260926 19:34:07.255874 15737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:07.338284 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 15614
I20260926 19:34:07.354403 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15253
I20260926 19:34:07.355628 15315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:07.461961 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 15253
I20260926 19:34:07.482366 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40257
--webserver_interface=127.6.215.254
--webserver_port=34631
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:40257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:07.808858 15759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:07.809160 15759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:07.809247 15759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:07.816629 15759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:34:07.816744 15759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:07.816845 15759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:07.816929 15759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:34:07.816973 15759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:34:07.827045 15759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:40257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=34631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:07.828356 15759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:07.829691 15759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:07.841533 15765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:07.841590 15764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:07.841549 15767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:07.842506 15759 server_base.cc:1034] running on GCE node
I20260926 19:34:07.843276 15759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:07.844681 15759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:07.845904 15759 hybrid_clock.cc:648] HybridClock initialized: now 1790451247845852 us; error 44 us; skew 500 ppm
I20260926 19:34:07.846246 15759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:07 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451247, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:07.849413 15759 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:34:07.850808 15759 webserver.cc:460] Webserver started at http://127.6.215.254:34631/ using document root <none> and password file <none>
I20260926 19:34:07.851504 15759 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:07.851657 15759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:07.857111 15759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260926 19:34:07.860051 15774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:07.861527 15759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:34:07.861703 15759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2679468b5c264c839c2841d16ad8183f"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "073dd422b5cbe9af71b92cc32dc54f27"
server_key_iv: "4d8003cb187ce7128158b335252ae3c5"
server_key_version: "encryptionkey@0"
I20260926 19:34:07.862702 15759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:07.877929 15759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:07.882471 15759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:07.882735 15759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:07.900810 15759 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:40257
I20260926 19:34:07.900815 15825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:40257 every 8 connection(s)
I20260926 19:34:07.902415 15759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:34:07.903967 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 15759
I20260926 19:34:07.905004 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:46181
--local_ip_for_outbound_sockets=127.6.215.193
--tserver_master_addrs=127.6.215.254:40257
--webserver_port=44879
--webserver_interface=127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:34:07.912032 15826 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:34:07.915511 15826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Bootstrap starting.
I20260926 19:34:07.934609 15826 log.cc:824] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:07.943149 15826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:34:07.943667 15826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Bootstrap complete.
I20260926 19:34:07.952632 15826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:07.953521 15826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2679468b5c264c839c2841d16ad8183f, State: Initialized, Role: FOLLOWER
I20260926 19:34:07.954216 15826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:07.954445 15826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:34:07.954558 15826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:34:07.954739 15826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:34:07.958552 15826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:07.959044 15826 leader_election.cc:302] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2679468b5c264c839c2841d16ad8183f; no voters:
I20260926 19:34:07.959673 15826 leader_election.cc:288] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 19:34:07.959808 15830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:34:07.962067 15826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:34:07.962066 15830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [term 2 LEADER]: Becoming Leader. State: Replica: 2679468b5c264c839c2841d16ad8183f, State: Running, Role: LEADER
I20260926 19:34:07.962766 15830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } }
I20260926 19:34:07.965353 15832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2679468b5c264c839c2841d16ad8183f. Latest consensus state: current_term: 2 leader_uuid: "2679468b5c264c839c2841d16ad8183f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } } }
I20260926 19:34:07.965720 15832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:07.965525 15831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2679468b5c264c839c2841d16ad8183f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2679468b5c264c839c2841d16ad8183f" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40257 } } }
I20260926 19:34:07.965859 15831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:07.976131 15839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:34:07.981935 15839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:34:07.983446 15839 catalog_manager.cc:1259] Loaded cluster ID: 2a586095111c45b5835f775363ccdfd4
I20260926 19:34:07.983620 15839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:34:07.987546 15839 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:34:07.989142 15839 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2679468b5c264c839c2841d16ad8183f: Loaded TSK: 0
I20260926 19:34:07.990784 15839 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 19:34:08.252198 15828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:08.252548 15828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:08.252700 15828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:08.260538 15828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:08.260733 15828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:08.260958 15828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:34:08.271940 15828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.193.keytab
--principal=oryx/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:46181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=44879
--tserver_master_addrs=127.6.215.254:40257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:08.273411 15828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:08.274760 15828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:08.289865 15852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:08.290062 15853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:08.291122 15855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:08.291180 15828 server_base.cc:1034] running on GCE node
I20260926 19:34:08.291829 15828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:08.292856 15828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:08.294091 15828 hybrid_clock.cc:648] HybridClock initialized: now 1790451248294022 us; error 59 us; skew 500 ppm
I20260926 19:34:08.294440 15828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:08 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451248, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:08.297616 15828 init.cc:377] Logged in from keytab as oryx/127.6.215.193@KRBTEST.COM (short username oryx)
I20260926 19:34:08.299278 15828 webserver.cc:460] Webserver started at http://127.6.215.193:44879/ using document root <none> and password file <none>
I20260926 19:34:08.300025 15828 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:08.300184 15828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:08.305879 15828 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 19:34:08.309032 15862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:08.310494 15828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:34:08.310671 15828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "437dc7e438254da1bdf82fc851f97e16"
format_stamp: "Formatted at 2026-09-26 19:34:05 on dist-test-slave-phbh"
server_key: "93896180af0a4d350d87b6963151024f"
server_key_iv: "20c18898e36b2a37d5b78760b5fba674"
server_key_version: "encryptionkey@0"
I20260926 19:34:08.311637 15828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:08.328133 15828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:08.333454 15828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:08.333762 15828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:08.334831 15828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:08.336292 15828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:08.336407 15828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:08.336529 15828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:08.336613 15828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:08.368080 15828 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:46181
I20260926 19:34:08.368134 15974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:46181 every 8 connection(s)
I20260926 19:34:08.369541 15828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:34:08.371905 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 15828
I20260926 19:34:08.372645 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:42835
--local_ip_for_outbound_sockets=127.6.215.194
--tserver_master_addrs=127.6.215.254:40257
--webserver_port=41945
--webserver_interface=127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:34:08 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:08 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 19:34:08.720139 15979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:08.720549 15979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:08.720711 15979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:08.732988 15979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:08.733139 15979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:08.733343 15979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:34:08.747968 15979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.194.keytab
--principal=oryx/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:42835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=41945
--tserver_master_addrs=127.6.215.254:40257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:08.749671 15979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:08.751276 15979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:08.769299 15986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:08.770104 15988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:08.770574 15979 server_base.cc:1034] running on GCE node
W20260926 19:34:08.771257 15985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:08.772017 15979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:08.773042 15979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:08.774255 15979 hybrid_clock.cc:648] HybridClock initialized: now 1790451248774198 us; error 54 us; skew 500 ppm
I20260926 19:34:08.774606 15979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:08 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451248, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:08.777876 15979 init.cc:377] Logged in from keytab as oryx/127.6.215.194@KRBTEST.COM (short username oryx)
I20260926 19:34:08.779445 15979 webserver.cc:460] Webserver started at http://127.6.215.194:41945/ using document root <none> and password file <none>
I20260926 19:34:08.780167 15979 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:08.780296 15979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:08.785626 15979 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:34:08.788225 15995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:08.789587 15979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:34:08.789743 15979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "3cb6bbf1570a43a3ad9e4ecae8a48eb1"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "c46737d16c7c3eff9e6c1461696c2bcf"
server_key_iv: "25f0322dfbada8c4c36b945d71f7590c"
server_key_version: "encryptionkey@0"
I20260926 19:34:08.790674 15979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:08.807080 15979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:08.811684 15979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:08.811952 15979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:08.813030 15979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:08.814507 15979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:08.814587 15979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:08.814716 15979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:08.814787 15979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:08.856810 15979 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:42835
I20260926 19:34:08.856863 16107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:42835 every 8 connection(s)
I20260926 19:34:08.858201 15979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:34:08.866470 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 15979
I20260926 19:34:08.867517 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:46617
--local_ip_for_outbound_sockets=127.6.215.195
--tserver_master_addrs=127.6.215.254:40257
--webserver_port=37003
--webserver_interface=127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:34:08 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:08 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 19:34:09.212741 16112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:09.213116 16112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:09.213281 16112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:09.221848 16112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:09.221992 16112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:09.222177 16112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:34:09.232194 16112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/krb5kdc/oryx_127.6.215.195.keytab
--principal=oryx/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:46617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=37003
--tserver_master_addrs=127.6.215.254:40257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:09.233712 16112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:09.235296 16112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:09.253907 16118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:09.255592 16119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:09.260175 16121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:09.260639 16112 server_base.cc:1034] running on GCE node
I20260926 19:34:09.261320 16112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:09.262482 16112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:09.263775 16112 hybrid_clock.cc:648] HybridClock initialized: now 1790451249263699 us; error 61 us; skew 500 ppm
I20260926 19:34:09.264168 16112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:09 dist-test-slave-phbh krb5kdc[15240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451249, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:09.267854 16112 init.cc:377] Logged in from keytab as oryx/127.6.215.195@KRBTEST.COM (short username oryx)
I20260926 19:34:09.270128 16112 webserver.cc:460] Webserver started at http://127.6.215.195:37003/ using document root <none> and password file <none>
I20260926 19:34:09.270932 16112 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:09.271097 16112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:09.277199 16112 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 19:34:09.280049 16128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:09.281912 16112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:34:09.282112 16112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "9e38e1d97a6342078fed53662dab8053"
format_stamp: "Formatted at 2026-09-26 19:34:06 on dist-test-slave-phbh"
server_key: "ccc1a4d12215f71ce98fd4ba92b3b02f"
server_key_iv: "cfe0cf2bc1af4d2071f02f12f89e00aa"
server_key_version: "encryptionkey@0"
I20260926 19:34:09.283367 16112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:09.300097 16112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:09.305415 16112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:09.305707 16112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:09.306849 16112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:09.308409 16112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:09.308527 16112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:09.308676 16112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:09.308755 16112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:09.357551 16112 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:46617
I20260926 19:34:09.357596 16240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:46617 every 8 connection(s)
I20260926 19:34:09.359164 16112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:34:09.365762 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 16112
Sep 26 19:34:09 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:09 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:09.405424 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:09.367670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.6.215.254:40257)
0926 19:34:09.372528 (+ 4858us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:09.372533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:09.372549 (+ 16us) server_negotiation.cc:407] Connection header received
0926 19:34:09.372599 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:09.372603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:09.372667 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:09.372832 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:09.375619 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.377639 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.380063 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.385651 (+ 5588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.386692 (+ 1041us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:09.391375 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:09.391384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:09.391400 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:09.391436 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:09.397534 (+ 6098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:09.398435 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:09.398445 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:09.398456 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:09.398675 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:09.401253 (+ 2578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:09.401262 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:09.401268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:09.401948 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:09.402282 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:09.403826 (+ 1544us) server_negotiation.cc:299] Negotiation successful
0926 19:34:09.404323 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4893,"server-negotiator.queue_time_us":4740,"thread_start_us":4639,"threads_started":1}
W20260926 19:34:09.973601 15978 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.590s user 0.006s sys 0.008s
W20260926 19:34:09.975875 15977 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:08.372606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:48863 (local address 127.6.215.254:40257)
0926 19:34:08.373427 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:08.373445 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:08.373922 (+ 477us) server_negotiation.cc:407] Connection header received
0926 19:34:08.375985 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:08.375998 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:08.376385 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:08.376803 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:08.378375 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.380267 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.381830 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.382463 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.382666 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:09.975366 (+1592700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:48863: BlockingRecv error: recv got EOF from 127.6.215.193:48863 (error 108)
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":372,"threads_started":1}
I20260926 19:34:09.977869 15978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:08.372866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:48863)
0926 19:34:08.373747 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:08.373784 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:08.375445 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:08.376950 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:08.376962 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:08.377676 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:08.378186 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:08.378212 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.380417 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.380427 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:08.381690 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:08.381703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.382696 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.382704 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:08.382975 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:08.383620 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:08.383643 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:09.976157 (+1592514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":444,"threads_started":1}
W20260926 19:34:09.980000 15975 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:40257 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.215.254:40257: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Sep 26 19:34:09 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:09 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
W20260926 19:34:10.001608 15977 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:09.981388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:57727 (local address 127.6.215.254:40257)
0926 19:34:09.981647 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:09.981658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:09.981967 (+ 309us) server_negotiation.cc:407] Connection header received
0926 19:34:09.982313 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:09.982322 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:09.982439 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:09.982546 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:09.986072 (+ 3526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.988269 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.991114 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.992197 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.992386 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.001180 (+ 8794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:57727: BlockingRecv error: recv got EOF from 127.6.215.193:57727 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:34:10.002679 15978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:09.981150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:57727)
0926 19:34:09.981842 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:09.981871 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:09.982117 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:09.982918 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:09.982927 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:09.985463 (+ 2536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:09.985789 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:09.985807 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.988551 (+ 2744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.988560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:09.990894 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:09.990916 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.992480 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.992490 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:09.992754 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:09.994027 (+ 1273us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:09.994062 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.002202 (+ 8140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229}
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:10.023389 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:10.003941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:33233)
0926 19:34:10.004903 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:10.004931 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:10.005115 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:10.005664 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:10.005672 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:10.006493 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:10.006906 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.006923 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.009184 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.009193 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.011485 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.011507 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.013815 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.013826 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.014145 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.015645 (+ 1500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:10.015684 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.022832 (+ 7148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":456,"threads_started":1}
W20260926 19:34:10.023525 15977 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:10.004006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:33233 (local address 127.6.215.254:40257)
0926 19:34:10.004255 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:10.004264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:10.005063 (+ 799us) server_negotiation.cc:407] Connection header received
0926 19:34:10.005314 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:10.005324 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:10.005461 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:10.005596 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:10.007113 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.008976 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.011985 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.013263 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.013408 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.023156 (+ 9748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:33233: BlockingRecv error: recv got EOF from 127.6.215.193:33233 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 19:34:10.024230 15975 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 19:34:10.329217 16111 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.458s user 0.004s sys 0.011s
W20260926 19:34:10.330868 16110 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:08.861153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:42081 (local address 127.6.215.254:40257)
0926 19:34:08.861585 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:08.861590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:08.863461 (+ 1871us) server_negotiation.cc:407] Connection header received
0926 19:34:08.865414 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:08.865420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:08.865522 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:08.865644 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:08.867278 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.868141 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.869704 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.870109 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.870238 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.330618 (+1460380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:42081: BlockingRecv error: recv got EOF from 127.6.215.194:42081 (error 108)
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":229,"threads_started":1}
I20260926 19:34:10.332051 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:08.862226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:42081)
0926 19:34:08.863307 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:08.863351 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:08.865149 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:08.865883 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:08.865891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:08.866500 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:08.867122 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:08.867138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.868295 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.868299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:08.869564 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:08.869576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:08.870333 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:08.870342 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:08.870571 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:08.871255 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:08.871286 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.330468 (+1459182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":468,"threads_started":1}
W20260926 19:34:10.333177 16108 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:40257 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.215.254:40257: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:10.351020 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:10.333842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:33245)
0926 19:34:10.334022 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:10.334039 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:10.334126 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:10.334603 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:10.334606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:10.335052 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:10.335242 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.335254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.337407 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.337411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.338369 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.338380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.339662 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.339667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.339785 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.346053 (+ 6268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:10.346074 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.350813 (+ 4739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 19:34:10.351665 16110 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:10.333946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:33245 (local address 127.6.215.254:40257)
0926 19:34:10.334203 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:10.334212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:10.334241 (+ 29us) server_negotiation.cc:407] Connection header received
0926 19:34:10.334325 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:10.334333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:10.334441 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:10.334597 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:10.335438 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.337236 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.338516 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.339468 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.339644 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.351287 (+ 11643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:33245: BlockingRecv error: recv got EOF from 127.6.215.194:33245 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:10.360383 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:10.351737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:49273)
0926 19:34:10.352091 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:10.352110 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:10.352224 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:10.352562 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:10.352566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:10.352951 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:10.353100 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.353108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.354146 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.354151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.355100 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.355110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.355821 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.355827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.356016 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.356621 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:10.356638 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.360105 (+ 3467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251}
W20260926 19:34:10.360548 15977 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:10.351912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:49273 (local address 127.6.215.254:40257)
0926 19:34:10.352093 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:10.352100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:10.352183 (+ 83us) server_negotiation.cc:407] Connection header received
0926 19:34:10.352355 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:10.352360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:10.352431 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:10.352529 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:10.353233 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.354000 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.355245 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.355666 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.355796 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.360303 (+ 4507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:49273: BlockingRecv error: recv got EOF from 127.6.215.194:49273 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260926 19:34:10.361048 16108 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 19:34:10.635986 16243 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.242s user 0.005s sys 0.010s
W20260926 19:34:10.637204 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:09.365139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:44921 (local address 127.6.215.254:40257)
0926 19:34:09.365663 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:09.365670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:09.366479 (+ 809us) server_negotiation.cc:407] Connection header received
0926 19:34:09.375473 (+ 8994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:09.375478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:09.375550 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:09.375659 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:09.380687 (+ 5028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.382301 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.386978 (+ 4677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.387729 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.387836 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.636941 (+1249105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:44921: BlockingRecv error: recv got EOF from 127.6.215.195:44921 (error 108)
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":399,"thread_start_us":2339,"threads_started":1}
I20260926 19:34:10.637684 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:09.364052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:44921)
0926 19:34:09.365205 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:09.365287 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:09.375201 (+ 9914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:09.376319 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:09.376332 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:09.377240 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:09.380479 (+ 3239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:09.380511 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.382541 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.382551 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:09.385457 (+ 2906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:09.385479 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:09.392315 (+ 6836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:09.392323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:09.392592 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:09.393835 (+ 1243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:09.393869 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.636666 (+1242797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":3358,"threads_started":1}
W20260926 19:34:10.638520 16241 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:40257 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.215.254:40257: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:10.647929 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:10.639174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:47937)
0926 19:34:10.639370 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:10.639387 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:10.639477 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:10.639769 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:10.639774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:10.640151 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:10.640307 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.640315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.641387 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.641391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.642347 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.642357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.643045 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.643052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.643232 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.643833 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:10.643849 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.647701 (+ 3852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260926 19:34:10.648039 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:10.639249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:47937 (local address 127.6.215.254:40257)
0926 19:34:10.639418 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:10.639422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:10.639438 (+ 16us) server_negotiation.cc:407] Connection header received
0926 19:34:10.639579 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:10.639582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:10.639636 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:10.639712 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:10.640432 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.641263 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.642498 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.642877 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.642965 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.647877 (+ 4912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:47937: BlockingRecv error: recv got EOF from 127.6.215.195:47937 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:10 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:10.656602 16243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:10.648703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:36509)
0926 19:34:10.648873 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:10.648889 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:10.649012 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:10.649342 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:10.649347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:10.649649 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:10.649790 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.649797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.650791 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.650796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.651728 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:10.651737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.652491 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.652497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:10.652677 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.653330 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:10.653346 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:10.656349 (+ 3003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260926 19:34:10.656720 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:10.648754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:36509 (local address 127.6.215.254:40257)
0926 19:34:10.649063 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:10.649071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:10.649089 (+ 18us) server_negotiation.cc:407] Connection header received
0926 19:34:10.649135 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:10.649140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:10.649216 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:10.649321 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:10.649929 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.650652 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.651913 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:10.652337 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:10.652469 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:10.656495 (+ 4026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:36509: BlockingRecv error: recv got EOF from 127.6.215.195:36509 (error 108)
Metrics: {"server-negotiator.queue_time_us":204}
W20260926 19:34:10.657075 16241 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 19:34:11 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:11 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:11.033516 16249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:11.024906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:40173)
0926 19:34:11.025386 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:11.025412 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:11.025518 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:11.025878 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:11.025882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:11.026215 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:11.026360 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:11.026366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.027370 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.027374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:11.028279 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:11.028289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.029050 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.029054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:11.029164 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:11.029788 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:11.029805 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:11.033304 (+ 3499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":275,"threads_started":1}
W20260926 19:34:11.033716 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:11.025015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:40173 (local address 127.6.215.254:40257)
0926 19:34:11.025195 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:11.025200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:11.025486 (+ 286us) server_negotiation.cc:407] Connection header received
0926 19:34:11.025611 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:11.025615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:11.025738 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:11.025829 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:11.026471 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.027244 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.028417 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.028914 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.029007 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:11.033468 (+ 4461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:40173: BlockingRecv error: recv got EOF from 127.6.215.193:40173 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 19:34:11 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:11 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:11.371086 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:11.361619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:33611)
0926 19:34:11.362181 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:11.362196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:11.362302 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:11.362609 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:11.362612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:11.363139 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:11.363281 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:11.363289 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.364272 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.364276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:11.365415 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:11.365426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.366238 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.366247 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:11.366381 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:11.367045 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:11.367067 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:11.370864 (+ 3797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":353,"threads_started":1}
W20260926 19:34:11.371311 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:11.361723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:33611 (local address 127.6.215.254:40257)
0926 19:34:11.361933 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:11.361941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:11.362267 (+ 326us) server_negotiation.cc:407] Connection header received
0926 19:34:11.362426 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:11.362429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:11.362487 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:11.362569 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:11.363416 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.364146 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.365536 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.366069 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.366161 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:11.371057 (+ 4896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:33611: BlockingRecv error: recv got EOF from 127.6.215.194:33611 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 19:34:11 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:11 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:11.666680 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:11.657712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:40837)
0926 19:34:11.658279 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:11.658296 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:11.658409 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:11.658786 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:11.658791 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:11.659272 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:11.659421 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:11.659431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.660391 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.660398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:11.661421 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:11.661432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.662190 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.662196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:11.662380 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:11.662977 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:11.662993 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:11.666447 (+ 3454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":334,"threads_started":1}
W20260926 19:34:11.666838 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:11.657824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:40837 (local address 127.6.215.254:40257)
0926 19:34:11.658027 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:11.658034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:11.658386 (+ 352us) server_negotiation.cc:407] Connection header received
0926 19:34:11.658553 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:11.658557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:11.658636 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:11.658756 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:11.659564 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.660254 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.661560 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:11.662009 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:11.662161 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:11.666617 (+ 4456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:40837: BlockingRecv error: recv got EOF from 127.6.215.195:40837 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 26 19:34:12 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:12 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:12.043143 16252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:12.034444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:50217)
0926 19:34:12.034959 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:12.034975 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:12.035085 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:12.035375 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:12.035379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:12.035716 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:12.035864 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:12.035872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.036871 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.036875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:12.037797 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:12.037808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.038415 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.038419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:12.038535 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:12.039170 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:12.039187 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:12.042903 (+ 3716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":282,"threads_started":1}
W20260926 19:34:12.043347 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:12.034508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:50217 (local address 127.6.215.254:40257)
0926 19:34:12.034686 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:12.034691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:12.035048 (+ 357us) server_negotiation.cc:407] Connection header received
0926 19:34:12.035178 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:12.035181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:12.035238 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:12.035334 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:12.035985 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.036694 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.037915 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.038295 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.038388 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:12.043078 (+ 4690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:50217: BlockingRecv error: recv got EOF from 127.6.215.193:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 19:34:12 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:12 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:12.381281 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:12.372012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:49129)
0926 19:34:12.372540 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:12.372556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:12.372669 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:12.373063 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:12.373068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:12.373485 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:12.373674 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:12.373682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.374637 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.374641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:12.375578 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:12.375588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.376279 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.376283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:12.376417 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:12.377029 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:12.377091 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:12.381030 (+ 3939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":307,"threads_started":1}
W20260926 19:34:12.381420 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:12.372084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:49129 (local address 127.6.215.254:40257)
0926 19:34:12.372293 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:12.372303 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:12.372636 (+ 333us) server_negotiation.cc:407] Connection header received
0926 19:34:12.372793 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:12.372797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:12.372888 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:12.373010 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:12.373802 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.374507 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.375696 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.376142 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.376233 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:12.381213 (+ 4980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:49129: BlockingRecv error: recv got EOF from 127.6.215.194:49129 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 19:34:12 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:12 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:12.676689 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:12.667636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:43355)
0926 19:34:12.668135 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:12.668156 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:12.668320 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:12.668759 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:12.668795 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:12.669190 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:12.669368 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:12.669376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.670533 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.670540 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:12.671465 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:12.671476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.672345 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.672351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:12.672528 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:12.673098 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:12.673116 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:12.676445 (+ 3329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":266,"threads_started":1}
W20260926 19:34:12.676971 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:12.667907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:43355 (local address 127.6.215.254:40257)
0926 19:34:12.668124 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:12.668130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:12.668248 (+ 118us) server_negotiation.cc:407] Connection header received
0926 19:34:12.668450 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:12.668455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:12.668546 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:12.668633 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:12.669526 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.670408 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.671603 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:12.672193 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:12.672328 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:12.676657 (+ 4329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:43355: BlockingRecv error: recv got EOF from 127.6.215.195:43355 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 19:34:13 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:13 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:13.052753 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:13.044027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:50705)
0926 19:34:13.044483 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:13.044498 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:13.044646 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:13.044990 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:13.044995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:13.045379 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:13.045540 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:13.045549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.046563 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.046573 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:13.047489 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:13.047500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.048273 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.048278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:13.048462 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:13.049102 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:13.049118 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:13.052420 (+ 3302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":220,"threads_started":1}
W20260926 19:34:13.052867 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:13.044073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:50705 (local address 127.6.215.254:40257)
0926 19:34:13.044295 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:13.044301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:13.044577 (+ 276us) server_negotiation.cc:407] Connection header received
0926 19:34:13.044762 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:13.044788 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:13.044864 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:13.044961 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:13.045678 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.046417 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.047634 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.048122 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.048250 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:13.052615 (+ 4365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:50705: BlockingRecv error: recv got EOF from 127.6.215.193:50705 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 26 19:34:13 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:13 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:13.391026 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:13.382197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:35237)
0926 19:34:13.382691 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:13.382707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:13.382819 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:13.383121 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:13.383124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:13.383451 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:13.383610 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:13.383618 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.384607 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.384611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:13.385637 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:13.385648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.386282 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.386285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:13.386401 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:13.387028 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:13.387046 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:13.390809 (+ 3763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":269,"threads_started":1}
W20260926 19:34:13.391321 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:13.382257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:35237 (local address 127.6.215.254:40257)
0926 19:34:13.382427 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:13.382431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:13.382778 (+ 347us) server_negotiation.cc:407] Connection header received
0926 19:34:13.382935 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:13.382939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:13.382996 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:13.383073 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:13.383755 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.384478 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.385761 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.386167 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.386260 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:13.391026 (+ 4766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:35237: BlockingRecv error: recv got EOF from 127.6.215.194:35237 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 26 19:34:13 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:13 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:13.685823 16257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:13.677674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:36095)
0926 19:34:13.678158 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:13.678174 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:13.678281 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:13.678563 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:13.678566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:13.678902 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:13.679064 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:13.679072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.679990 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.679994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:13.680978 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:13.680990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.681599 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.681603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:13.681720 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:13.682280 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:13.682299 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:13.685579 (+ 3280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":261,"threads_started":1}
W20260926 19:34:13.685992 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:13.677753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:36095 (local address 127.6.215.254:40257)
0926 19:34:13.677968 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:13.677975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:13.678253 (+ 278us) server_negotiation.cc:407] Connection header received
0926 19:34:13.678375 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:13.678379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:13.678446 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:13.678523 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:13.679179 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.679872 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.681091 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:13.681487 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:13.681575 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:13.685759 (+ 4184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:36095: BlockingRecv error: recv got EOF from 127.6.215.195:36095 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 19:34:14 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:14 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:14.063171 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:14.053789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:58151)
0926 19:34:14.054265 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:14.054281 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:14.054453 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:14.054781 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:14.054786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:14.055183 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:14.055339 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:14.055347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.056327 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.056333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:14.057341 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:14.057351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.058056 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.058061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:14.058230 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:14.058964 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:14.058985 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:14.062927 (+ 3942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":251,"threads_started":1}
W20260926 19:34:14.063367 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:14.053858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:58151 (local address 127.6.215.254:40257)
0926 19:34:14.054086 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:14.054094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:14.054384 (+ 290us) server_negotiation.cc:407] Connection header received
0926 19:34:14.054582 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:14.054586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:14.054643 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:14.054743 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:14.055482 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.056185 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.057488 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.057907 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.058033 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:14.063124 (+ 5091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:58151: BlockingRecv error: recv got EOF from 127.6.215.193:58151 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 19:34:14 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:14 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:14.401381 16259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:14.392086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:37931)
0926 19:34:14.392593 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:14.392609 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:14.392725 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:14.393044 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:14.393047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:14.393468 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:14.393686 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:14.393701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.394801 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.394805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:14.395986 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:14.395998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.396932 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.396940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:14.397128 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:14.397707 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:14.397723 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:14.401021 (+ 3298us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":305,"threads_started":1}
W20260926 19:34:14.401432 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:14.392148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:37931 (local address 127.6.215.254:40257)
0926 19:34:14.392326 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:14.392336 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:14.392684 (+ 348us) server_negotiation.cc:407] Connection header received
0926 19:34:14.392861 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:14.392864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:14.392921 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:14.393003 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:14.393816 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.394691 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.396162 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.396742 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.396882 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:14.401208 (+ 4326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:37931: BlockingRecv error: recv got EOF from 127.6.215.194:37931 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 19:34:14 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:14 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:14.695401 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:14.686708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:45495)
0926 19:34:14.687265 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:14.687282 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:14.687398 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:14.687689 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:14.687692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:14.688076 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:14.688219 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:14.688227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.689241 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.689244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:14.690228 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:14.690239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.690901 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.690905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:14.691020 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:14.691563 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:14.691578 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:14.695057 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":289,"threads_started":1}
W20260926 19:34:14.695581 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:14.686813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:45495 (local address 127.6.215.254:40257)
0926 19:34:14.686987 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:14.686992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:14.687380 (+ 388us) server_negotiation.cc:407] Connection header received
0926 19:34:14.687497 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:14.687501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:14.687577 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:14.687676 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:14.688371 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.689128 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.690367 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:14.690778 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:14.690879 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:14.695288 (+ 4409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:45495: BlockingRecv error: recv got EOF from 127.6.215.195:45495 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 19:34:15 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:15 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:15.073153 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:15.064131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:39785)
0926 19:34:15.064736 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:15.064752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:15.064938 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:15.065274 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:15.065279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:15.065731 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:15.065891 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:15.065900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.066981 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.066989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:15.067987 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:15.067998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.068664 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.068671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:15.068864 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:15.069399 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:15.069416 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:15.072823 (+ 3407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":370,"threads_started":1}
W20260926 19:34:15.073289 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:15.064244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:39785 (local address 127.6.215.254:40257)
0926 19:34:15.064587 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:15.064593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:15.064866 (+ 273us) server_negotiation.cc:407] Connection header received
0926 19:34:15.065062 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:15.065068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:15.065134 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:15.065239 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:15.066033 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.066844 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.068127 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.068506 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.068631 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:15.073054 (+ 4423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:39785: BlockingRecv error: recv got EOF from 127.6.215.193:39785 (error 108)
Metrics: {"server-negotiator.queue_time_us":187}
Sep 26 19:34:15 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:15 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:15.415617 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:15.402377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:47093)
0926 19:34:15.402871 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:15.402886 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:15.403030 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:15.403371 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:15.403378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:15.403835 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:15.403980 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:15.403988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.405074 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.405080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:15.406027 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:15.406037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.406756 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.406763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:15.406943 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:15.411078 (+ 4135us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:15.411097 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:15.415256 (+ 4159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":259,"threads_started":1}
W20260926 19:34:15.416157 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:15.402529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:47093 (local address 127.6.215.254:40257)
0926 19:34:15.402738 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:15.402744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:15.402966 (+ 222us) server_negotiation.cc:407] Connection header received
0926 19:34:15.403165 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:15.403170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:15.403238 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:15.403338 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:15.404126 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.404953 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.406166 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.406597 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.406727 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:15.415903 (+ 9176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:47093: BlockingRecv error: recv got EOF from 127.6.215.194:47093 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 26 19:34:15 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:15 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:15.705085 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:15.696515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:49965)
0926 19:34:15.697023 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:15.697040 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:15.697153 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:15.697444 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:15.697447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:15.697869 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:15.698009 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:15.698017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.698935 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.698939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:15.700033 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:15.700045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.700650 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.700654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:15.700828 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:15.701450 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:15.701466 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:15.704863 (+ 3397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":287,"threads_started":1}
W20260926 19:34:15.705168 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:15.696656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:49965 (local address 127.6.215.254:40257)
0926 19:34:15.696907 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:15.696913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:15.697120 (+ 207us) server_negotiation.cc:407] Connection header received
0926 19:34:15.697248 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:15.697251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:15.697324 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:15.697404 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:15.698128 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.698820 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.700150 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:15.700532 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:15.700619 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:15.705004 (+ 4385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:49965: BlockingRecv error: recv got EOF from 127.6.215.195:49965 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 26 19:34:16 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:16 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:16.082670 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:16.074217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:41235)
0926 19:34:16.074750 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:16.074771 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:16.074912 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:16.075288 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:16.075292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:16.075672 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:16.075808 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:16.075815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.076827 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.076830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:16.077733 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:16.077743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.078505 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.078509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:16.078626 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:16.079205 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:16.079221 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:16.082452 (+ 3231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":257,"threads_started":1}
W20260926 19:34:16.082888 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:16.074300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:41235 (local address 127.6.215.254:40257)
0926 19:34:16.074526 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:16.074535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:16.074865 (+ 330us) server_negotiation.cc:407] Connection header received
0926 19:34:16.075041 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:16.075049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:16.075146 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:16.075223 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:16.075943 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.076681 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.077865 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.078368 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.078508 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:16.082652 (+ 4144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:41235: BlockingRecv error: recv got EOF from 127.6.215.193:41235 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 19:34:16 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:16 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:16.426225 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:16.417395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:40229)
0926 19:34:16.417956 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:16.417980 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:16.418102 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:16.418413 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:16.418416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:16.418792 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:16.418939 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:16.418946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.419887 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.419891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:16.420921 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:16.420933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.421646 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.421651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:16.421775 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:16.422359 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:16.422375 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:16.425936 (+ 3561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":345,"threads_started":1}
W20260926 19:34:16.426458 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:16.417470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:40229 (local address 127.6.215.254:40257)
0926 19:34:16.417676 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:16.417681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:16.418062 (+ 381us) server_negotiation.cc:407] Connection header received
0926 19:34:16.418204 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:16.418208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:16.418265 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:16.418356 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:16.419072 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.419741 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.421060 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.421486 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.421578 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:16.426144 (+ 4566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:40229: BlockingRecv error: recv got EOF from 127.6.215.194:40229 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Sep 26 19:34:16 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:16 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:16.714864 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:16.705995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:37423)
0926 19:34:16.706501 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:16.706518 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:16.706624 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:16.706940 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:16.706944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:16.707341 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:16.707494 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:16.707502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.708653 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.708656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:16.709655 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:16.709666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.710355 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.710359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:16.710488 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:16.711036 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:16.711053 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:16.714516 (+ 3463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":296,"threads_started":1}
W20260926 19:34:16.715031 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:16.706105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:37423 (local address 127.6.215.254:40257)
0926 19:34:16.706273 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:16.706279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:16.706595 (+ 316us) server_negotiation.cc:407] Connection header received
0926 19:34:16.706721 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:16.706725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:16.706802 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:16.706878 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:16.707700 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.708513 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.709806 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:16.710211 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:16.710308 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:16.714715 (+ 4407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:37423: BlockingRecv error: recv got EOF from 127.6.215.195:37423 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 19:34:17 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:17 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:17.093003 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:17.083558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:58927)
0926 19:34:17.084173 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:17.084188 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:17.084354 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:17.084707 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:17.084712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:17.085174 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:17.085346 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:17.085354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.086398 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.086404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:17.087345 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:17.087356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.088176 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.088182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:17.088348 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:17.089014 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:17.089032 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:17.092676 (+ 3644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":278,"threads_started":1}
W20260926 19:34:17.093143 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:17.083724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:58927 (local address 127.6.215.254:40257)
0926 19:34:17.083964 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:17.083973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:17.084297 (+ 324us) server_negotiation.cc:407] Connection header received
0926 19:34:17.084487 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:17.084492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:17.084558 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:17.084676 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:17.085495 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.086229 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.087496 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.088008 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.088148 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:17.092929 (+ 4781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:58927: BlockingRecv error: recv got EOF from 127.6.215.193:58927 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 26 19:34:17 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:17 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:17.437217 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:17.427251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:39379)
0926 19:34:17.427935 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:17.427954 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:17.428142 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:17.428540 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:17.428545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:17.429104 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:17.429300 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:17.429310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.430563 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.430569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:17.431647 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:17.431660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.432465 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.432471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:17.432659 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:17.433401 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:17.433421 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:17.436953 (+ 3532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":291,"threads_started":1}
W20260926 19:34:17.437353 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:17.427464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:39379 (local address 127.6.215.254:40257)
0926 19:34:17.427743 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:17.427750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:17.428064 (+ 314us) server_negotiation.cc:407] Connection header received
0926 19:34:17.428301 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:17.428307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:17.428385 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:17.428506 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:17.429468 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.430406 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.431813 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.432289 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.432446 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:17.437124 (+ 4678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:39379: BlockingRecv error: recv got EOF from 127.6.215.194:39379 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Sep 26 19:34:17 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:17 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:17.725682 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:17.715897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:57463)
0926 19:34:17.716540 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:17.716556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:17.716716 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:17.717338 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:17.717343 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:17.717816 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:17.717989 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:17.717998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.719040 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.719046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:17.720184 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:17.720196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.721059 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.721063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:17.721263 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:17.722029 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:17.722046 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:17.725436 (+ 3390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":305,"threads_started":1}
W20260926 19:34:17.725813 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:17.715989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:57463 (local address 127.6.215.254:40257)
0926 19:34:17.716203 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:17.716213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:17.716646 (+ 433us) server_negotiation.cc:407] Connection header received
0926 19:34:17.717090 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:17.717098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:17.717188 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:17.717310 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:17.718165 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.718879 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.720339 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:17.720907 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:17.721033 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:17.725606 (+ 4573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:57463: BlockingRecv error: recv got EOF from 127.6.215.195:57463 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 19:34:18 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:18 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:18.103459 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:18.094089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:45991)
0926 19:34:18.094667 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:18.094682 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:18.094872 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:18.095346 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:18.095353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:18.095872 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:18.096116 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:18.096129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.097182 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.097186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:18.098125 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:18.098137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.098799 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.098803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:18.098910 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:18.099495 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:18.099511 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:18.103105 (+ 3594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":310,"threads_started":1}
W20260926 19:34:18.103639 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:18.094370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:45991 (local address 127.6.215.254:40257)
0926 19:34:18.094571 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:18.094577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:18.094795 (+ 218us) server_negotiation.cc:407] Connection header received
0926 19:34:18.095011 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:18.095019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:18.095094 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:18.095176 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:18.096266 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.097056 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.098249 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.098645 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.098737 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:18.103354 (+ 4617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:45991: BlockingRecv error: recv got EOF from 127.6.215.193:45991 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 26 19:34:18 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:18 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:18.453064 16271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:18.438228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:50649)
0926 19:34:18.438696 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:18.438712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:18.438827 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:18.439119 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:18.439122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:18.439526 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:18.439692 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:18.439700 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.440918 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.440922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:18.441846 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:18.441857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.442546 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.442550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:18.442662 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:18.449135 (+ 6473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:18.449155 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:18.452716 (+ 3561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":246,"threads_started":1}
W20260926 19:34:18.453239 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:18.438374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:50649 (local address 127.6.215.254:40257)
0926 19:34:18.438641 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:18.438648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:18.438794 (+ 146us) server_negotiation.cc:407] Connection header received
0926 19:34:18.438931 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:18.438934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:18.438994 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:18.439072 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:18.439837 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.440727 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.441973 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.442381 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.442471 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:18.453021 (+ 10550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:50649: BlockingRecv error: recv got EOF from 127.6.215.194:50649 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Sep 26 19:34:18 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:18 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:18.734961 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:18.726811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:41201)
0926 19:34:18.727289 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:18.727305 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:18.727410 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:18.727691 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:18.727694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:18.728086 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:18.728237 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:18.728247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.729288 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.729291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:18.730191 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:18.730201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.730858 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.730862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:18.730973 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:18.731532 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:18.731548 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:18.734705 (+ 3157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":244,"threads_started":1}
W20260926 19:34:18.735162 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:18.726838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:41201 (local address 127.6.215.254:40257)
0926 19:34:18.727034 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:18.727043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:18.727389 (+ 346us) server_negotiation.cc:407] Connection header received
0926 19:34:18.727508 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:18.727512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:18.727569 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:18.727647 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:18.728372 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.729169 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.730318 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:18.730710 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:18.730797 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:18.734899 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:41201: BlockingRecv error: recv got EOF from 127.6.215.195:41201 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 26 19:34:19 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:19 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:19.113507 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:19.104496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:55897)
0926 19:34:19.105039 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:19.105055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:19.105159 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:19.105522 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:19.105526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:19.105892 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:19.106027 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:19.106034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.107121 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.107124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:19.108029 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:19.108039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.108660 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.108664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:19.108862 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:19.109415 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:19.109432 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:19.113279 (+ 3847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":315,"threads_started":1}
W20260926 19:34:19.113663 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:19.104611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:55897 (local address 127.6.215.254:40257)
0926 19:34:19.104850 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:19.104854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:19.105130 (+ 276us) server_negotiation.cc:407] Connection header received
0926 19:34:19.105308 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:19.105317 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:19.105379 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:19.105465 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:19.106192 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.107001 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.108151 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.108534 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.108622 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:19.113457 (+ 4835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:55897: BlockingRecv error: recv got EOF from 127.6.215.193:55897 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 19:34:19 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:19 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:19.463982 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:19.454083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:39257)
0926 19:34:19.454634 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:19.454650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:19.454779 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:19.455070 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:19.455074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:19.455450 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:19.455592 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:19.455600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.456609 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.456613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:19.457639 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:19.457652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.458556 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.458560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:19.458701 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:19.459418 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:19.459436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:19.463725 (+ 4289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
W20260926 19:34:19.464181 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:19.454152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:39257 (local address 127.6.215.254:40257)
0926 19:34:19.454334 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:19.454338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:19.454726 (+ 388us) server_negotiation.cc:407] Connection header received
0926 19:34:19.454866 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:19.454869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:19.454923 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:19.455002 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:19.455717 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.456466 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.457970 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.458409 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.458516 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:19.463917 (+ 5401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:39257: BlockingRecv error: recv got EOF from 127.6.215.194:39257 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 19:34:19 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:19 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:19.744666 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:19.736024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:56841)
0926 19:34:19.736585 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:19.736602 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:19.736707 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:19.737013 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:19.737016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:19.737405 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:19.737563 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:19.737572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.738502 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.738506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:19.739520 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:19.739533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.740480 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.740486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:19.740668 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:19.741291 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:19.741307 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:19.744428 (+ 3121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":320,"threads_started":1}
W20260926 19:34:19.744761 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:19.736108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:56841 (local address 127.6.215.254:40257)
0926 19:34:19.736365 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:19.736369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:19.736680 (+ 311us) server_negotiation.cc:407] Connection header received
0926 19:34:19.736811 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:19.736815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:19.736873 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:19.736962 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:19.737694 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.738379 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.739689 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:19.740307 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:19.740441 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:19.744598 (+ 4157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:56841: BlockingRecv error: recv got EOF from 127.6.215.195:56841 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Sep 26 19:34:20 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:20 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:20.123296 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:20.114636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:58909)
0926 19:34:20.115100 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:20.115116 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:20.115268 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:20.115621 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:20.115626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:20.115997 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:20.116142 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:20.116150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.117255 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.117260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:20.118238 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:20.118250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.118990 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.118996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:20.119165 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:20.119760 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:20.119777 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:20.123072 (+ 3295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":250,"threads_started":1}
W20260926 19:34:20.123543 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:20.114690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:58909 (local address 127.6.215.254:40257)
0926 19:34:20.114938 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:20.114947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:20.115199 (+ 252us) server_negotiation.cc:407] Connection header received
0926 19:34:20.115407 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:20.115412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:20.115484 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:20.115588 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:20.116304 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.117106 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.118383 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.118834 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.118977 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:20.123267 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:58909: BlockingRecv error: recv got EOF from 127.6.215.193:58909 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 19:34:20 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:20 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:20.479748 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:20.464963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:38929)
0926 19:34:20.465478 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:20.465494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:20.465668 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:20.466086 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:20.466091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:20.466460 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:20.466607 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:20.466615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.467774 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.467780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:20.468734 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:20.468745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.469458 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.469464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:20.469633 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:20.475319 (+ 5686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:20.475375 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:20.479465 (+ 4090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":302,"threads_started":1}
W20260926 19:34:20.479856 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:20.465009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:38929 (local address 127.6.215.254:40257)
0926 19:34:20.465167 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:20.465171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:20.465597 (+ 426us) server_negotiation.cc:407] Connection header received
0926 19:34:20.465838 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:20.465843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:20.465916 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:20.466022 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:20.466786 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.467616 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.468903 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.469301 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.469443 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:20.479676 (+ 10233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:38929: BlockingRecv error: recv got EOF from 127.6.215.194:38929 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 26 19:34:20 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:20 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:20.755074 16278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:20.745613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:35869)
0926 19:34:20.746153 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:20.746172 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:20.746351 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:20.746721 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:20.746726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:20.747128 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:20.747289 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:20.747298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.748322 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.748329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:20.749337 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:20.749348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.750090 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.750096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:20.750276 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:20.750954 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:20.750971 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:20.754842 (+ 3871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":284,"threads_started":1}
W20260926 19:34:20.755316 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:20.745850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:35869 (local address 127.6.215.254:40257)
0926 19:34:20.746109 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:20.746117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:20.746282 (+ 165us) server_negotiation.cc:407] Connection header received
0926 19:34:20.746485 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:20.746490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:20.746574 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:20.746691 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:20.747448 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.748165 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.749479 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:20.749923 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:20.750058 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:20.755024 (+ 4966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:35869: BlockingRecv error: recv got EOF from 127.6.215.195:35869 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Sep 26 19:34:21 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:21 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:21.133769 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:21.124529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:45065)
0926 19:34:21.125051 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:21.125067 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:21.125234 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:21.125567 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:21.125572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:21.126010 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:21.126264 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:21.126278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.127541 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.127548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:21.128642 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:21.128654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.129341 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.129345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:21.129460 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:21.130008 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:21.130024 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:21.133449 (+ 3425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":291,"threads_started":1}
W20260926 19:34:21.133858 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:21.124624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:45065 (local address 127.6.215.254:40257)
0926 19:34:21.124924 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:21.124929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:21.125154 (+ 225us) server_negotiation.cc:407] Connection header received
0926 19:34:21.125359 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:21.125371 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:21.125430 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:21.125538 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:21.126448 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.127393 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.128792 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.129203 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.129294 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:21.133626 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:45065: BlockingRecv error: recv got EOF from 127.6.215.193:45065 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 26 19:34:21 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:21 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:21.490108 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:21.480847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:48137)
0926 19:34:21.481351 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:21.481369 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:21.481535 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:21.481889 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:21.481893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:21.482346 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:21.482514 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:21.482523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.483550 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.483558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:21.484478 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:21.484488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.485405 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.485410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:21.485594 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:21.486260 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:21.486287 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:21.489901 (+ 3614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":259,"threads_started":1}
W20260926 19:34:21.490355 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:21.481088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:48137 (local address 127.6.215.254:40257)
0926 19:34:21.481308 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:21.481315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:21.481469 (+ 154us) server_negotiation.cc:407] Connection header received
0926 19:34:21.481678 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:21.481683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:21.481750 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:21.481857 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:21.482667 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.483412 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.484628 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.485242 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.485375 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:21.490079 (+ 4704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:48137: BlockingRecv error: recv got EOF from 127.6.215.194:48137 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 26 19:34:21 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:21 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:21.765420 16281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:21.755940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:36601)
0926 19:34:21.756426 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:21.756442 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:21.756608 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:21.757020 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:21.757023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:21.757419 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:21.757691 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:21.757702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.758743 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.758747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:21.759651 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:21.759661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.760440 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.760444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:21.760561 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:21.761355 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:21.761377 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:21.765197 (+ 3820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":281,"threads_started":1}
W20260926 19:34:21.765674 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:21.756102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:36601 (local address 127.6.215.254:40257)
0926 19:34:21.756310 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:21.756316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:21.756540 (+ 224us) server_negotiation.cc:407] Connection header received
0926 19:34:21.756739 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:21.756746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:21.756863 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:21.756979 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:21.757843 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.758626 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.759774 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:21.760306 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:21.760398 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:21.765407 (+ 5009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:36601: BlockingRecv error: recv got EOF from 127.6.215.195:36601 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 26 19:34:22 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:22 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:22.144189 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:22.134805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:55775)
0926 19:34:22.135300 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:22.135318 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:22.135490 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:22.135853 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:22.135856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:22.136230 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:22.136387 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:22.136394 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.137598 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.137602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:22.138517 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:22.138527 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.139233 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.139237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:22.139360 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:22.139953 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:22.139969 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:22.143850 (+ 3881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":393,"threads_started":1}
W20260926 19:34:22.144399 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:22.134925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:55775 (local address 127.6.215.254:40257)
0926 19:34:22.135189 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:22.135196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:22.135413 (+ 217us) server_negotiation.cc:407] Connection header received
0926 19:34:22.135626 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:22.135633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:22.135721 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:22.135804 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:22.136624 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.137453 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.138645 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.139109 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.139199 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:22.144107 (+ 4908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:55775: BlockingRecv error: recv got EOF from 127.6.215.193:55775 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 19:34:22 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:22 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:22.500517 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:22.491159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:51955)
0926 19:34:22.491612 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:22.491628 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:22.491742 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:22.492029 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:22.492032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:22.492426 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:22.492586 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:22.492594 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.493645 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.493649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:22.495066 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:22.495080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.495848 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.495853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:22.496036 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:22.496749 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:22.496773 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:22.500147 (+ 3374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":256,"threads_started":1}
W20260926 19:34:22.500567 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:22.491265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:51955 (local address 127.6.215.254:40257)
0926 19:34:22.491514 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:22.491521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:22.491707 (+ 186us) server_negotiation.cc:407] Connection header received
0926 19:34:22.491843 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:22.491847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:22.491907 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:22.491988 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:22.492725 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.493518 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.495220 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.495690 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.495836 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:22.500334 (+ 4498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:51955: BlockingRecv error: recv got EOF from 127.6.215.194:51955 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Sep 26 19:34:22 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:22 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:22.775352 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:22.766430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:60755)
0926 19:34:22.767033 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:22.767049 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:22.767196 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:22.767524 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:22.767527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:22.767997 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:22.768153 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:22.768161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.769312 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.769316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:22.770216 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:22.770227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.770974 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.770978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:22.771103 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:22.771774 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:22.771791 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:22.775100 (+ 3309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":323,"threads_started":1}
W20260926 19:34:22.775614 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:22.766456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:60755 (local address 127.6.215.254:40257)
0926 19:34:22.766774 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:22.766779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:22.767143 (+ 364us) server_negotiation.cc:407] Connection header received
0926 19:34:22.767314 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:22.767318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:22.767398 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:22.767488 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:22.768296 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.769186 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.770381 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:22.770819 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:22.770922 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:22.775310 (+ 4388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:60755: BlockingRecv error: recv got EOF from 127.6.215.195:60755 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Sep 26 19:34:23 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:23 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:23.155052 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:23.145241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:40497)
0926 19:34:23.145652 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:23.145667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:23.145825 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:23.146313 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:23.146318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:23.146863 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:23.147018 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:23.147027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.148480 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.148486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:23.149839 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:23.149851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.150496 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.150500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:23.150626 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:23.151257 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:23.151274 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:23.154718 (+ 3444us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":216,"threads_started":1}
W20260926 19:34:23.155210 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:23.145249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:40497 (local address 127.6.215.254:40257)
0926 19:34:23.145498 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:23.145505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:23.145758 (+ 253us) server_negotiation.cc:407] Connection header received
0926 19:34:23.145993 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:23.145998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:23.146103 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:23.146216 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:23.147157 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.148323 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.149970 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.150371 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.150468 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:23.154946 (+ 4478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:40497: BlockingRecv error: recv got EOF from 127.6.215.193:40497 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 19:34:23 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:23 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.194@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:23.510722 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:23.501658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.194:38007)
0926 19:34:23.502180 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:23.502196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:23.502307 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:23.502653 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:23.502657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:23.503032 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:23.503174 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:23.503182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.504116 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.504120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:23.505056 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:23.505069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.505825 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.505832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:23.505962 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:23.506727 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:23.506749 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:23.510492 (+ 3743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":304,"threads_started":1}
W20260926 19:34:23.510957 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:23.501801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:38007 (local address 127.6.215.254:40257)
0926 19:34:23.501970 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:23.501975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:23.502274 (+ 299us) server_negotiation.cc:407] Connection header received
0926 19:34:23.502414 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:23.502417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:23.502475 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:23.502554 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:23.503302 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.503986 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.505190 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.505639 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.505730 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:23.510689 (+ 4959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.194:38007: BlockingRecv error: recv got EOF from 127.6.215.194:38007 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 19:34:23 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:23 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.195@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:23.785262 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:23.776308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.195:59607)
0926 19:34:23.776807 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:23.776823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:23.776930 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:23.777218 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:23.777221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:23.777620 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:23.777775 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:23.777783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.778741 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.778744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:23.779620 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:23.779630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.780260 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.780264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:23.780378 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:23.780998 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:23.781014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:23.784903 (+ 3889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":260,"threads_started":1}
W20260926 19:34:23.785336 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:23.776360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:59607 (local address 127.6.215.254:40257)
0926 19:34:23.776541 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:23.776545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:23.776903 (+ 358us) server_negotiation.cc:407] Connection header received
0926 19:34:23.777025 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:23.777028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:23.777087 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:23.777169 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:23.777894 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.778605 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.779743 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:23.780137 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:23.780234 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:23.785106 (+ 4872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.195:59607: BlockingRecv error: recv got EOF from 127.6.215.195:59607 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 19:34:24 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
Sep 26 19:34:24 dist-test-slave-phbh krb5kdc[15240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.215.193@KRBTEST.COM for oryx/127.6.215.254@KRBTEST.COM, Server not found in Kerberos database
I20260926 19:34:24.164916 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:24.156392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40257 (local address 127.6.215.193:48321)
0926 19:34:24.156954 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:24.156972 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:24.157079 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:24.157349 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:24.157353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:24.157756 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:24.157919 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:24.157928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.158889 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.158893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:24.159800 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:24.159811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.160464 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.160467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:24.160595 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:24.161208 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:24.161225 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:24.164586 (+ 3361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.215.254:40257: Server oryx/127.6.215.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":276,"threads_started":1}
W20260926 19:34:24.165103 16244 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:24.156471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:48321 (local address 127.6.215.254:40257)
0926 19:34:24.156808 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:24.156816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:24.157049 (+ 233us) server_negotiation.cc:407] Connection header received
0926 19:34:24.157163 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:24.157166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:24.157226 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:24.157306 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:24.158044 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.158747 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.159920 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.160340 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.160430 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:24.164836 (+ 4406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.215.193:48321: BlockingRecv error: recv got EOF from 127.6.215.193:48321 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
I20260926 19:34:24.368570 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15828
I20260926 19:34:24.382051 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:24.369711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.6.215.193:46181)
0926 19:34:24.370388 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:24.370395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:24.370423 (+ 28us) server_negotiation.cc:407] Connection header received
0926 19:34:24.370490 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:24.370496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:24.370809 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:24.370988 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:24.372890 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.374036 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.375202 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.375766 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.375861 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:24.378712 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:24.378722 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:24.378736 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:24.378789 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:24.381524 (+ 2735us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.6.215.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 19:34:24.381826 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43246: Request ticket server kudu/127.6.215.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":338,"threads_started":1}
W20260926 19:34:24.382184 16290 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43246: Request ticket server kudu/127.6.215.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 19:34:24.382292 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15828: Not authorized: Client connection negotiation failed: client connection to 127.6.215.193:46181: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.6.215.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 19:34:24.382530 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 15828
I20260926 19:34:24.400099 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15979
I20260926 19:34:24.412575 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:24.401272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60258 (local address 127.6.215.194:42835)
0926 19:34:24.401965 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:24.401975 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:24.402005 (+ 30us) server_negotiation.cc:407] Connection header received
0926 19:34:24.402083 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:24.402095 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:24.402413 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:24.402634 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:24.403832 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.404958 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.406114 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.406650 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.406815 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:24.409398 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:24.409407 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:24.409420 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:24.409472 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:24.412069 (+ 2597us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.6.215.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 19:34:24.412338 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60258: Request ticket server kudu/127.6.215.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":348,"threads_started":1}
W20260926 19:34:24.412729 16291 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60258: Request ticket server kudu/127.6.215.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 19:34:24.412892 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15979: Not authorized: Client connection negotiation failed: client connection to 127.6.215.194:42835: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.6.215.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 19:34:24.413026 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 15979
I20260926 19:34:24.430601 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16112
I20260926 19:34:24.443243 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:24.431865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.6.215.195:46617)
0926 19:34:24.432453 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:24.432460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:24.432486 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:34:24.432562 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:24.432570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:24.432879 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:24.433102 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:24.434511 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.435533 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.436845 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:24.437453 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:24.437630 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:24.440185 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:24.440201 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:24.440217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:24.440266 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:24.442724 (+ 2458us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.6.215.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 19:34:24.442994 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60476: Request ticket server kudu/127.6.215.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":303,"threads_started":1}
W20260926 19:34:24.443373 16292 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60476: Request ticket server kudu/127.6.215.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 19:34:24.443400 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16112: Not authorized: Client connection negotiation failed: client connection to 127.6.215.195:46617: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.6.215.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 19:34:24.443572 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 16112
I20260926 19:34:24.456841 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 15759
I20260926 19:34:24.458252 15821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:24.572898 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 15759
2026-09-26T19:34:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21137 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T19:34:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:34:24Z Disabled control of system clock
I20260926 19:34:24.648037 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:39291
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:39291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 19:34:24.967795 16297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:24.968173 16297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:24.968299 16297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:24.975504 16297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:34:24.975669 16297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:24.975766 16297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:24.975875 16297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:34:24.975953 16297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:34:24.985105 16297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:39291
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:39291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:24.986428 16297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:24.988215 16297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:25.000612 16303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:25.000798 16305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:25.001147 16297 server_base.cc:1034] running on GCE node
W20260926 19:34:25.001392 16302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:25.002260 16297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:25.003664 16297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:25.004875 16297 hybrid_clock.cc:648] HybridClock initialized: now 1790451265004821 us; error 38 us; skew 500 ppm
I20260926 19:34:25.005235 16297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:25.007555 16297 webserver.cc:460] Webserver started at http://127.6.215.254:35015/ using document root <none> and password file <none>
I20260926 19:34:25.008265 16297 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:25.008400 16297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:25.008728 16297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:25.011618 16297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "b6d9adb795e14f2cac0edbe99b968abb"
format_stamp: "Formatted at 2026-09-26 19:34:25 on dist-test-slave-phbh"
server_key: "50bddffb92be26a0e055c04b1ccc3253"
server_key_iv: "a1a0166bf9e0468d6e49d49f14e953bb"
server_key_version: "encryptionkey@0"
I20260926 19:34:25.012431 16297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "b6d9adb795e14f2cac0edbe99b968abb"
format_stamp: "Formatted at 2026-09-26 19:34:25 on dist-test-slave-phbh"
server_key: "50bddffb92be26a0e055c04b1ccc3253"
server_key_iv: "a1a0166bf9e0468d6e49d49f14e953bb"
server_key_version: "encryptionkey@0"
I20260926 19:34:25.017782 16297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 19:34:25.021175 16311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:25.022565 16297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 19:34:25.022771 16297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "b6d9adb795e14f2cac0edbe99b968abb"
format_stamp: "Formatted at 2026-09-26 19:34:25 on dist-test-slave-phbh"
server_key: "50bddffb92be26a0e055c04b1ccc3253"
server_key_iv: "a1a0166bf9e0468d6e49d49f14e953bb"
server_key_version: "encryptionkey@0"
I20260926 19:34:25.023000 16297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:25.037773 16297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:25.038636 16297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:25.038936 16297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:25.057680 16297 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:39291
I20260926 19:34:25.057708 16362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:39291 every 8 connection(s)
I20260926 19:34:25.059144 16297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:34:25.062922 16363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:25.068627 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 16297
I20260926 19:34:25.068848 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:34:25.069512 7007 external_mini_cluster.cc:1428] Setting key 7a97f5d1b8940c8aca7fea6136e61879
I20260926 19:34:25.073618 16363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb: Bootstrap starting.
I20260926 19:34:25.077044 16363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:25.078354 16363 log.cc:824] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:25.081233 16363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb: No bootstrap required, opened a new log
I20260926 19:34:25.083674 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:25.071650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35236 (local address 127.6.215.254:39291)
0926 19:34:25.072654 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:25.072665 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:25.072714 (+ 49us) server_negotiation.cc:407] Connection header received
0926 19:34:25.074475 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:25.074497 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:25.074975 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:25.075304 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:25.075974 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:25.077350 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:25.079271 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:25.079962 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:25.080167 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:25.081384 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:25.081393 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:25.081407 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:25.081956 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:25.082030 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:25.082050 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:25.082188 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:25.082412 (+ 224us) server_negotiation.cc:299] Negotiation successful
0926 19:34:25.082722 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":376,"threads_started":1}
I20260926 19:34:25.089879 16363 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d9adb795e14f2cac0edbe99b968abb" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39291 } }
I20260926 19:34:25.090335 16363 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:25.090468 16363 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d9adb795e14f2cac0edbe99b968abb, State: Initialized, Role: FOLLOWER
I20260926 19:34:25.091015 16363 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d9adb795e14f2cac0edbe99b968abb" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39291 } }
I20260926 19:34:25.091254 16363 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:34:25.091387 16363 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:34:25.091526 16363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:25.094128 16363 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d9adb795e14f2cac0edbe99b968abb" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39291 } }
I20260926 19:34:25.094594 16363 leader_election.cc:302] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d9adb795e14f2cac0edbe99b968abb; no voters:
I20260926 19:34:25.095130 16363 leader_election.cc:288] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:34:25.095295 16368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:25.096339 16368 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [term 1 LEADER]: Becoming Leader. State: Replica: b6d9adb795e14f2cac0edbe99b968abb, State: Running, Role: LEADER
I20260926 19:34:25.096890 16368 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d9adb795e14f2cac0edbe99b968abb" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39291 } }
I20260926 19:34:25.097308 16363 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:34:25.099502 16370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d9adb795e14f2cac0edbe99b968abb. Latest consensus state: current_term: 1 leader_uuid: "b6d9adb795e14f2cac0edbe99b968abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d9adb795e14f2cac0edbe99b968abb" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39291 } } }
I20260926 19:34:25.099869 16370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:25.101035 16369 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d9adb795e14f2cac0edbe99b968abb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d9adb795e14f2cac0edbe99b968abb" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 39291 } } }
I20260926 19:34:25.101431 16369 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:25.103850 16377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:34:25.107789 16377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:34:25.115790 16377 catalog_manager.cc:1347] Generated new cluster ID: 0cb8f50dd2e74c2dbb8e6c55389a60b1
I20260926 19:34:25.115906 16377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:34:25.143895 16377 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:34:25.145282 16377 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:34:25.170055 16377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d9adb795e14f2cac0edbe99b968abb: Generated new TSK 0
I20260926 19:34:25.170953 16377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:34:25.183563 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39291
--builtin_ntp_servers=127.6.215.212:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 19:34:25.522722 16387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:25.523010 16387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:25.523151 16387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:25.530918 16387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:25.531019 16387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:25.531191 16387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:34:25.540562 16387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:25.541923 16387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:25.543781 16387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:25.557371 16393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:25.557308 16395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:25.557364 16392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:25.558395 16387 server_base.cc:1034] running on GCE node
I20260926 19:34:25.559024 16387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:25.559995 16387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:25.561236 16387 hybrid_clock.cc:648] HybridClock initialized: now 1790451265561185 us; error 47 us; skew 500 ppm
I20260926 19:34:25.561569 16387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:25.567646 16387 webserver.cc:460] Webserver started at http://127.6.215.193:45617/ using document root <none> and password file <none>
I20260926 19:34:25.568454 16387 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:25.568603 16387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:25.569023 16387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:25.571987 16387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "f9e9901643534d36ab14fafadb071c10"
format_stamp: "Formatted at 2026-09-26 19:34:25 on dist-test-slave-phbh"
server_key: "f1b7d10ba09fb77e3be30a75955fb45b"
server_key_iv: "f6b5c091a1ff1bcec71af25f051b02c8"
server_key_version: "encryptionkey@0"
I20260926 19:34:25.572860 16387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "f9e9901643534d36ab14fafadb071c10"
format_stamp: "Formatted at 2026-09-26 19:34:25 on dist-test-slave-phbh"
server_key: "f1b7d10ba09fb77e3be30a75955fb45b"
server_key_iv: "f6b5c091a1ff1bcec71af25f051b02c8"
server_key_version: "encryptionkey@0"
I20260926 19:34:25.577991 16387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 19:34:25.581602 16401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:25.582957 16387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 19:34:25.583144 16387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "f9e9901643534d36ab14fafadb071c10"
format_stamp: "Formatted at 2026-09-26 19:34:25 on dist-test-slave-phbh"
server_key: "f1b7d10ba09fb77e3be30a75955fb45b"
server_key_iv: "f6b5c091a1ff1bcec71af25f051b02c8"
server_key_version: "encryptionkey@0"
I20260926 19:34:25.583372 16387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:25.608918 16387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:25.609802 16387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:25.610121 16387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:25.611071 16387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:25.612524 16387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:25.612663 16387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:25.612828 16387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:25.612924 16387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 19:34:25.646222 16387 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:34189
I20260926 19:34:25.646317 16513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:34189 every 8 connection(s)
I20260926 19:34:25.647536 16387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:34:25.649745 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 16387
I20260926 19:34:25.649958 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:34:25.650787 7007 external_mini_cluster.cc:1428] Setting key db9dfb218ab59d5411c9205fbf759e71
I20260926 19:34:25.653843 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39291
--builtin_ntp_servers=127.6.215.212:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 19:34:25.662298 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:25.650093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:45753 (local address 127.6.215.254:39291)
0926 19:34:25.650583 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:25.650590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:25.651964 (+ 1374us) server_negotiation.cc:407] Connection header received
0926 19:34:25.654321 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:25.654325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:25.654386 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:25.654473 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:25.655904 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:25.656610 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:25.659131 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:25.659528 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:25.659620 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:25.661288 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:25.661296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:25.661303 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:25.661331 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:25.661392 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:25.661410 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:25.661508 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:25.661736 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 19:34:25.661849 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":235,"threads_started":1}
I20260926 19:34:25.663151 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:25.650686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39291 (local address 127.6.215.193:45753)
0926 19:34:25.651791 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:25.651845 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:25.654037 (+ 2192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:25.654740 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:25.654754 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:25.654889 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:25.655731 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:25.655759 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:25.656804 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:25.656812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:25.658981 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:25.658998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:25.659761 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:25.659770 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:25.660103 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:25.660891 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:34:25.660896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:25.660951 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:34:25.661059 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:25.661518 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:25.661526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:25.661532 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 19:34:25.661674 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 19:34:25.662034 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":329,"threads_started":1}
I20260926 19:34:25.664831 16514 heartbeater.cc:347] Connected to a master server at 127.6.215.254:39291
I20260926 19:34:25.665146 16514 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:25.666041 16514 heartbeater.cc:511] Master 127.6.215.254:39291 requested a full tablet report, sending...
I20260926 19:34:25.668253 16328 ts_manager.cc:194] Registered new tserver with Master: f9e9901643534d36ab14fafadb071c10 (127.6.215.193:34189)
I20260926 19:34:25.670116 16328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.193:45753
W20260926 19:34:25.982115 16518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:25.982506 16518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:25.982674 16518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:25.990291 16518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:25.990438 16518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:25.990618 16518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:34:26.001195 16518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:26.002514 16518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:26.004251 16518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:26.017624 16526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:26.017894 16523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:26.017922 16524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:26.018802 16518 server_base.cc:1034] running on GCE node
I20260926 19:34:26.019451 16518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:26.020414 16518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:26.021659 16518 hybrid_clock.cc:648] HybridClock initialized: now 1790451266021595 us; error 60 us; skew 500 ppm
I20260926 19:34:26.021975 16518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:26.024155 16518 webserver.cc:460] Webserver started at http://127.6.215.194:32937/ using document root <none> and password file <none>
I20260926 19:34:26.024900 16518 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:26.025027 16518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:26.025344 16518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:26.028242 16518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "aab5ed5ce7ed414d8091b90237e587bb"
format_stamp: "Formatted at 2026-09-26 19:34:26 on dist-test-slave-phbh"
server_key: "813e3a79bd80c4bc674ff4387c80f71d"
server_key_iv: "7fb92841bb5c59eb76ceb7758e0aeae9"
server_key_version: "encryptionkey@0"
I20260926 19:34:26.029064 16518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "aab5ed5ce7ed414d8091b90237e587bb"
format_stamp: "Formatted at 2026-09-26 19:34:26 on dist-test-slave-phbh"
server_key: "813e3a79bd80c4bc674ff4387c80f71d"
server_key_iv: "7fb92841bb5c59eb76ceb7758e0aeae9"
server_key_version: "encryptionkey@0"
I20260926 19:34:26.033967 16518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:34:26.037349 16532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:26.038730 16518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:34:26.038887 16518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "aab5ed5ce7ed414d8091b90237e587bb"
format_stamp: "Formatted at 2026-09-26 19:34:26 on dist-test-slave-phbh"
server_key: "813e3a79bd80c4bc674ff4387c80f71d"
server_key_iv: "7fb92841bb5c59eb76ceb7758e0aeae9"
server_key_version: "encryptionkey@0"
I20260926 19:34:26.039073 16518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:26.060388 16518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:26.061357 16518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:26.061697 16518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:26.062731 16518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:26.064297 16518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:26.064396 16518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:26.064559 16518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:26.064636 16518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:26.099619 16518 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:32881
I20260926 19:34:26.099730 16644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:32881 every 8 connection(s)
I20260926 19:34:26.101312 16518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:34:26.109695 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 16518
I20260926 19:34:26.109862 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:34:26.110291 7007 external_mini_cluster.cc:1428] Setting key ab14105397aaee964d65de1256aadd37
I20260926 19:34:26.112870 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39291
--builtin_ntp_servers=127.6.215.212:34229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 19:34:26.114451 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:26.103957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:49567 (local address 127.6.215.254:39291)
0926 19:34:26.104151 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:26.104156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:26.106027 (+ 1871us) server_negotiation.cc:407] Connection header received
0926 19:34:26.107835 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:26.107841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:26.107941 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:26.108082 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:26.109400 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.110251 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.111829 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.112214 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.112349 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:26.113689 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:26.113701 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:26.113705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:26.113721 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:26.113750 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:26.113761 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:26.113851 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:26.114082 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 19:34:26.114169 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:34:26.115458 16647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:26.104505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39291 (local address 127.6.215.194:49567)
0926 19:34:26.105705 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:26.105742 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:26.107567 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:26.108311 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:26.108319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:26.108406 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:26.109071 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:26.109090 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.110405 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.110410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:26.111677 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:26.111689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.112428 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.112435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:26.112653 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:26.113389 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:34:26.113391 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:26.113429 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:34:26.113510 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:26.113862 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:26.113873 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:26.113888 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 19:34:26.114059 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 19:34:26.114441 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":313,"threads_started":1}
I20260926 19:34:26.116974 16645 heartbeater.cc:347] Connected to a master server at 127.6.215.254:39291
I20260926 19:34:26.117281 16645 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:26.118039 16645 heartbeater.cc:511] Master 127.6.215.254:39291 requested a full tablet report, sending...
I20260926 19:34:26.119521 16328 ts_manager.cc:194] Registered new tserver with Master: aab5ed5ce7ed414d8091b90237e587bb (127.6.215.194:32881)
I20260926 19:34:26.120306 16328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.194:49567
W20260926 19:34:26.446344 16648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:26.446678 16648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:26.446836 16648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:26.454347 16648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:26.454499 16648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:26.454674 16648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:34:26.463804 16648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:39291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:26.465171 16648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:26.466908 16648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:26.480572 16654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:26.480548 16656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:26.480737 16653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:26.481482 16648 server_base.cc:1034] running on GCE node
I20260926 19:34:26.482089 16648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:26.483135 16648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:26.484380 16648 hybrid_clock.cc:648] HybridClock initialized: now 1790451266484340 us; error 62 us; skew 500 ppm
I20260926 19:34:26.484728 16648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:26.486981 16648 webserver.cc:460] Webserver started at http://127.6.215.195:42703/ using document root <none> and password file <none>
I20260926 19:34:26.487775 16648 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:26.487905 16648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:26.488214 16648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:26.491060 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "b1593844b4ab4f95a13928b247353463"
format_stamp: "Formatted at 2026-09-26 19:34:26 on dist-test-slave-phbh"
server_key: "1120afc75b89b808c23fe6d35f96587d"
server_key_iv: "35ad114c6aa6d07610a120d05b8efaf2"
server_key_version: "encryptionkey@0"
I20260926 19:34:26.491797 16648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "b1593844b4ab4f95a13928b247353463"
format_stamp: "Formatted at 2026-09-26 19:34:26 on dist-test-slave-phbh"
server_key: "1120afc75b89b808c23fe6d35f96587d"
server_key_iv: "35ad114c6aa6d07610a120d05b8efaf2"
server_key_version: "encryptionkey@0"
I20260926 19:34:26.496882 16648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260926 19:34:26.500507 16662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:26.502017 16648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:34:26.502157 16648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "b1593844b4ab4f95a13928b247353463"
format_stamp: "Formatted at 2026-09-26 19:34:26 on dist-test-slave-phbh"
server_key: "1120afc75b89b808c23fe6d35f96587d"
server_key_iv: "35ad114c6aa6d07610a120d05b8efaf2"
server_key_version: "encryptionkey@0"
I20260926 19:34:26.502341 16648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:26.524621 16648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:26.525542 16648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:26.525808 16648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:26.526746 16648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:26.528258 16648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:26.528342 16648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:26.528410 16648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:26.528465 16648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:26.560822 16648 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:38681
I20260926 19:34:26.560915 16774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:38681 every 8 connection(s)
I20260926 19:34:26.562188 16648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:34:26.566186 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 16648
I20260926 19:34:26.566370 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:34:26.566872 7007 external_mini_cluster.cc:1428] Setting key 3b0a85ed71a39222e815ccf975bc7257
I20260926 19:34:26.574651 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:26.564599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:55971 (local address 127.6.215.254:39291)
0926 19:34:26.564821 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:26.564827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:26.566192 (+ 1365us) server_negotiation.cc:407] Connection header received
0926 19:34:26.568087 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:26.568092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:26.568186 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:26.568295 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:26.569606 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.570318 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.571842 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.572304 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.572435 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:26.573800 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:26.573805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:26.573808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:26.573826 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:26.573855 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:26.573866 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:26.573961 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:26.574230 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 19:34:26.574320 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:34:26.575682 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:26.565169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:39291 (local address 127.6.215.195:55971)
0926 19:34:26.566023 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:26.566062 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:26.567791 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:26.568517 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:26.568526 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:26.568610 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:26.569429 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:26.569455 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.570469 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.570474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:26.571697 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:26.571709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.572529 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.572536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:26.572826 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:26.573470 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:34:26.573473 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:26.573512 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:34:26.573592 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:26.573972 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:26.573984 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:26.573992 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 19:34:26.574197 (+ 205us) client_negotiation.cc:240] Negotiation successful
0926 19:34:26.574587 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":250,"threads_started":1}
I20260926 19:34:26.577068 16775 heartbeater.cc:347] Connected to a master server at 127.6.215.254:39291
I20260926 19:34:26.577389 16775 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:26.578136 16775 heartbeater.cc:511] Master 127.6.215.254:39291 requested a full tablet report, sending...
I20260926 19:34:26.579507 16328 ts_manager.cc:194] Registered new tserver with Master: b1593844b4ab4f95a13928b247353463 (127.6.215.195:38681)
I20260926 19:34:26.580242 16328 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.195:55971
I20260926 19:34:26.582157 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 19:34:26.588212 16516 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:26.586667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.6.215.254:39291)
0926 19:34:26.586851 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:26.586855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:26.587332 (+ 477us) server_negotiation.cc:407] Connection header received
0926 19:34:26.587458 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:26.587461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:26.587535 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:26.587617 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:26.587923 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35004: BlockingRecv error: recv got EOF from 127.0.0.1:35004 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:34:26.588985 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16387
I20260926 19:34:26.596215 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:26.589850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.6.215.193:34189)
0926 19:34:26.590331 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:26.590338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:26.590363 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:34:26.590415 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:26.590421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:26.590624 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:26.590746 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:26.591064 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.592041 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.593304 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.593841 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.593942 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:26.594945 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:26.594952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:26.594964 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:26.595547 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:26.595590 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:26.595608 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:26.595705 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:26.595819 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 19:34:26.595924 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":233,"threads_started":1}
I20260926 19:34:26.597559 16509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:26.677765 16514 heartbeater.cc:503] Master 127.6.215.254:39291 was elected leader, sending a full tablet report...
I20260926 19:34:26.679270 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 16387
W20260926 19:34:26.697286 16315 connection.cc:535] server connection from 127.6.215.193:45753 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 19:34:26.697582 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16518
I20260926 19:34:26.705076 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:26.698466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.6.215.194:32881)
0926 19:34:26.699027 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:26.699034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:26.699061 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:34:26.699125 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:26.699132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:26.699349 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:26.699475 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:26.699836 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.700971 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.702206 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.702747 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.702841 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:26.703736 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:26.703744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:26.703757 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:26.704333 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:26.704377 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:26.704394 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:26.704480 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:26.704613 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 19:34:26.704729 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
I20260926 19:34:26.706295 16640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:26.786738 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 16518
I20260926 19:34:26.802968 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16648
I20260926 19:34:26.810756 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:26.803801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.6.215.195:38681)
0926 19:34:26.804411 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:26.804417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:26.804447 (+ 30us) server_negotiation.cc:407] Connection header received
0926 19:34:26.804506 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:26.804512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:26.804729 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:26.804935 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:26.805292 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.806434 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.807781 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:26.808321 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:26.808414 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:26.809407 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:26.809415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:26.809429 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:26.809988 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:26.810035 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:26.810053 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:26.810138 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:26.810257 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 19:34:26.810409 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":313,"threads_started":1}
I20260926 19:34:26.811937 16770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:26.891714 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 16648
I20260926 19:34:26.907537 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16297
I20260926 19:34:26.908720 16358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:27.008064 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 16297
2026-09-26T19:34:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2429 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T19:34:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:34:27Z Disabled control of system clock
I20260926 19:34:27.080134 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37753
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:37753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 19:34:27.412426 16794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:27.412952 16794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:27.413121 16794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:27.420734 16794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:34:27.420946 16794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:27.421026 16794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:27.421113 16794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:34:27.421176 16794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:34:27.430763 16794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:37753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:27.432176 16794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:27.434000 16794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:27.445844 16800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:27.445945 16802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:27.445914 16799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:27.447122 16794 server_base.cc:1034] running on GCE node
I20260926 19:34:27.447921 16794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:27.449488 16794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:27.450716 16794 hybrid_clock.cc:648] HybridClock initialized: now 1790451267450686 us; error 51 us; skew 500 ppm
I20260926 19:34:27.451081 16794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:27.453461 16794 webserver.cc:460] Webserver started at http://127.6.215.254:42919/ using document root <none> and password file <none>
I20260926 19:34:27.454183 16794 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:27.454315 16794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:27.454645 16794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:27.457724 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "d29d365d06e94ff8a5d36bbed59ebe75"
format_stamp: "Formatted at 2026-09-26 19:34:27 on dist-test-slave-phbh"
server_key: "1d2ba331b92544c5d0701907b1d66aed"
server_key_iv: "22b0e9e2034e29a4d92526343c8a7224"
server_key_version: "encryptionkey@0"
I20260926 19:34:27.458559 16794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "d29d365d06e94ff8a5d36bbed59ebe75"
format_stamp: "Formatted at 2026-09-26 19:34:27 on dist-test-slave-phbh"
server_key: "1d2ba331b92544c5d0701907b1d66aed"
server_key_iv: "22b0e9e2034e29a4d92526343c8a7224"
server_key_version: "encryptionkey@0"
I20260926 19:34:27.463802 16794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:34:27.467185 16808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:27.468576 16794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:34:27.468878 16794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "d29d365d06e94ff8a5d36bbed59ebe75"
format_stamp: "Formatted at 2026-09-26 19:34:27 on dist-test-slave-phbh"
server_key: "1d2ba331b92544c5d0701907b1d66aed"
server_key_iv: "22b0e9e2034e29a4d92526343c8a7224"
server_key_version: "encryptionkey@0"
I20260926 19:34:27.469107 16794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:27.475111 16794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:27.475888 16794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:27.476188 16794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:27.494390 16794 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:37753
I20260926 19:34:27.494421 16859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:37753 every 8 connection(s)
I20260926 19:34:27.495764 16794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:34:27.499481 16860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:27.503250 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 16794
I20260926 19:34:27.503466 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:34:27.504267 7007 external_mini_cluster.cc:1428] Setting key 3701891b930f6eeffa5a332d9bfc40c7
I20260926 19:34:27.512486 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:27.506324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.6.215.254:37753)
0926 19:34:27.507281 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:27.507292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:27.507332 (+ 40us) server_negotiation.cc:407] Connection header received
0926 19:34:27.508946 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:27.508969 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:27.509402 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:27.509802 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:27.510928 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:27.510933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:27.510945 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:27.510957 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:27.511009 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:27.511024 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:27.511257 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:27.511391 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 19:34:27.511626 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":332,"threads_started":1}
I20260926 19:34:27.512604 16860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75: Bootstrap starting.
I20260926 19:34:27.515957 16860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:27.517002 16860 log.cc:824] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:27.519905 16860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75: No bootstrap required, opened a new log
I20260926 19:34:27.527177 16860 raft_consensus.cc:348] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37753 } }
I20260926 19:34:27.527676 16860 raft_consensus.cc:374] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:27.527810 16860 raft_consensus.cc:729] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d29d365d06e94ff8a5d36bbed59ebe75, State: Initialized, Role: FOLLOWER
I20260926 19:34:27.528358 16860 consensus_queue.cc:260] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37753 } }
I20260926 19:34:27.528601 16860 raft_consensus.cc:388] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:34:27.528749 16860 raft_consensus.cc:482] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:34:27.528954 16860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:27.531087 16860 raft_consensus.cc:504] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37753 } }
I20260926 19:34:27.531595 16860 leader_election.cc:302] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d29d365d06e94ff8a5d36bbed59ebe75; no voters:
I20260926 19:34:27.532320 16860 leader_election.cc:288] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:34:27.532431 16865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:27.533138 16865 raft_consensus.cc:686] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [term 1 LEADER]: Becoming Leader. State: Replica: d29d365d06e94ff8a5d36bbed59ebe75, State: Running, Role: LEADER
I20260926 19:34:27.533748 16865 consensus_queue.cc:237] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37753 } }
I20260926 19:34:27.534638 16860 sys_catalog.cc:564] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:34:27.536556 16867 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d29d365d06e94ff8a5d36bbed59ebe75. Latest consensus state: current_term: 1 leader_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37753 } } }
I20260926 19:34:27.536947 16867 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:27.538882 16866 sys_catalog.cc:455] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d29d365d06e94ff8a5d36bbed59ebe75" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37753 } } }
I20260926 19:34:27.539157 16866 sys_catalog.cc:458] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75 [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:27.539155 16873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:34:27.544847 16873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:34:27.553865 16873 catalog_manager.cc:1347] Generated new cluster ID: bcf510ad01fb400bb3213458bbad8d04
I20260926 19:34:27.553996 16873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:34:27.573632 16873 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:34:27.574117 16873 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:34:27.585230 16873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d29d365d06e94ff8a5d36bbed59ebe75: Generated new TSK 0
I20260926 19:34:27.586354 16873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:34:27.596287 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:37753
--builtin_ntp_servers=127.6.215.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 19:34:27.920218 16884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:27.920490 16884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:27.920672 16884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:27.927837 16884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:27.927935 16884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:27.928123 16884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:34:27.937515 16884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:37753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:27.938810 16884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:27.940495 16884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:27.953392 16892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:27.953411 16889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:27.953641 16890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:27.954128 16884 server_base.cc:1034] running on GCE node
I20260926 19:34:27.954732 16884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:27.955673 16884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:27.956873 16884 hybrid_clock.cc:648] HybridClock initialized: now 1790451267956820 us; error 39 us; skew 500 ppm
I20260926 19:34:27.957253 16884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:27.959517 16884 webserver.cc:460] Webserver started at http://127.6.215.193:42717/ using document root <none> and password file <none>
I20260926 19:34:27.960166 16884 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:27.960249 16884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:27.960516 16884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:27.963299 16884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "d49ddfd18f34449c9cc8e6206471bfa7"
format_stamp: "Formatted at 2026-09-26 19:34:27 on dist-test-slave-phbh"
server_key: "451cffcd08ec6d69a0821b8cc1e1d7b7"
server_key_iv: "6bc9d2bcac38935882514b99a4e20c68"
server_key_version: "encryptionkey@0"
I20260926 19:34:27.964044 16884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "d49ddfd18f34449c9cc8e6206471bfa7"
format_stamp: "Formatted at 2026-09-26 19:34:27 on dist-test-slave-phbh"
server_key: "451cffcd08ec6d69a0821b8cc1e1d7b7"
server_key_iv: "6bc9d2bcac38935882514b99a4e20c68"
server_key_version: "encryptionkey@0"
I20260926 19:34:27.969081 16884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260926 19:34:27.972447 16898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:27.973841 16884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:34:27.974035 16884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "d49ddfd18f34449c9cc8e6206471bfa7"
format_stamp: "Formatted at 2026-09-26 19:34:27 on dist-test-slave-phbh"
server_key: "451cffcd08ec6d69a0821b8cc1e1d7b7"
server_key_iv: "6bc9d2bcac38935882514b99a4e20c68"
server_key_version: "encryptionkey@0"
I20260926 19:34:27.974224 16884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:27.979869 16884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:27.980602 16884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:27.980888 16884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:27.981747 16884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:27.983165 16884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:27.983249 16884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:27.983328 16884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:27.983382 16884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:28.015797 16884 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:36957
I20260926 19:34:28.015846 17010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:36957 every 8 connection(s)
I20260926 19:34:28.017204 16884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:34:28.018453 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 16884
I20260926 19:34:28.018685 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:34:28.019543 7007 external_mini_cluster.cc:1428] Setting key 6f36d5e722c647438aa831a6ebcbfd9d
I20260926 19:34:28.022701 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:37753
--builtin_ntp_servers=127.6.215.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 19:34:28.027575 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:28.020369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:60947 (local address 127.6.215.254:37753)
0926 19:34:28.020911 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:28.020915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:28.022450 (+ 1535us) server_negotiation.cc:407] Connection header received
0926 19:34:28.025234 (+ 2784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:28.025239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:28.025289 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:28.025382 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:28.026911 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:28.026915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:28.026918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:28.026936 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:28.026974 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:28.026986 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:28.027081 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:28.027306 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 19:34:28.027355 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":334,"threads_started":1}
I20260926 19:34:28.029104 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:28.021058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:37753 (local address 127.6.215.193:60947)
0926 19:34:28.022273 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:28.022324 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:28.024940 (+ 2616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:28.025673 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:28.025687 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:28.025819 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:28.026533 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:34:28.026538 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:28.026599 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:34:28.026729 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:28.027087 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:28.027100 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:28.027111 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 19:34:28.027390 (+ 279us) client_negotiation.cc:240] Negotiation successful
0926 19:34:28.027683 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":348,"threads_started":1}
I20260926 19:34:28.030685 17011 heartbeater.cc:347] Connected to a master server at 127.6.215.254:37753
I20260926 19:34:28.031008 17011 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:28.031594 17011 heartbeater.cc:511] Master 127.6.215.254:37753 requested a full tablet report, sending...
I20260926 19:34:28.033816 16825 ts_manager.cc:194] Registered new tserver with Master: d49ddfd18f34449c9cc8e6206471bfa7 (127.6.215.193:36957)
W20260926 19:34:28.357662 17015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:28.357941 17015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:28.358112 17015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:28.365489 17015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:28.365600 17015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:28.365803 17015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:34:28.375058 17015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:37753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:28.376358 17015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:28.378388 17015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:28.391564 17021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:28.391686 17023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:28.391700 17020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:28.392491 17015 server_base.cc:1034] running on GCE node
I20260926 19:34:28.393181 17015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:28.394207 17015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:28.395416 17015 hybrid_clock.cc:648] HybridClock initialized: now 1790451268395346 us; error 61 us; skew 500 ppm
I20260926 19:34:28.395741 17015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:28.398197 17015 webserver.cc:460] Webserver started at http://127.6.215.194:38057/ using document root <none> and password file <none>
I20260926 19:34:28.398905 17015 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:28.399022 17015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:28.399360 17015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:28.402309 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "1c5dbb0d234147e183aff1ac2e2b81d4"
format_stamp: "Formatted at 2026-09-26 19:34:28 on dist-test-slave-phbh"
server_key: "e5baca1d880862d6145a051714b9da2d"
server_key_iv: "61450022e07923ae1ce3f5ad7c8d377e"
server_key_version: "encryptionkey@0"
I20260926 19:34:28.403097 17015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "1c5dbb0d234147e183aff1ac2e2b81d4"
format_stamp: "Formatted at 2026-09-26 19:34:28 on dist-test-slave-phbh"
server_key: "e5baca1d880862d6145a051714b9da2d"
server_key_iv: "61450022e07923ae1ce3f5ad7c8d377e"
server_key_version: "encryptionkey@0"
I20260926 19:34:28.408264 17015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 19:34:28.411628 17029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:28.413021 17015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 19:34:28.413194 17015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "1c5dbb0d234147e183aff1ac2e2b81d4"
format_stamp: "Formatted at 2026-09-26 19:34:28 on dist-test-slave-phbh"
server_key: "e5baca1d880862d6145a051714b9da2d"
server_key_iv: "61450022e07923ae1ce3f5ad7c8d377e"
server_key_version: "encryptionkey@0"
I20260926 19:34:28.413416 17015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:28.418946 17015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:28.419703 17015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:28.419962 17015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:28.420930 17015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:28.422268 17015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:28.422392 17015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:28.422524 17015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:28.422612 17015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:28.455183 17015 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:44715
I20260926 19:34:28.455274 17141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:44715 every 8 connection(s)
I20260926 19:34:28.456492 17015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:34:28.459075 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 17015
I20260926 19:34:28.459263 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:34:28.459707 7007 external_mini_cluster.cc:1428] Setting key cf90e037a22248fc3e702f3d3e93f007
I20260926 19:34:28.462334 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:37753
--builtin_ntp_servers=127.6.215.212:37753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 19:34:28.465126 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:28.459279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:48697 (local address 127.6.215.254:37753)
0926 19:34:28.459476 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:28.459482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:28.461120 (+ 1638us) server_negotiation.cc:407] Connection header received
0926 19:34:28.462840 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:28.462848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:28.462925 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:28.463039 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:28.464320 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:28.464328 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:28.464336 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:28.464357 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:28.464398 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:28.464414 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:28.464531 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:28.464700 (+ 169us) server_negotiation.cc:299] Negotiation successful
0926 19:34:28.464762 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 19:34:28.465947 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:28.459853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:37753 (local address 127.6.215.194:48697)
0926 19:34:28.460942 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:28.460998 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:28.462555 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:28.463243 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:28.463251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:28.463339 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:28.464016 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:34:28.464018 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:28.464054 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:34:28.464135 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:28.464511 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:28.464517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:28.464523 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 19:34:28.464728 (+ 205us) client_negotiation.cc:240] Negotiation successful
0926 19:34:28.464992 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":333,"threads_started":1}
I20260926 19:34:28.467337 17142 heartbeater.cc:347] Connected to a master server at 127.6.215.254:37753
I20260926 19:34:28.467662 17142 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:28.468361 17142 heartbeater.cc:511] Master 127.6.215.254:37753 requested a full tablet report, sending...
I20260926 19:34:28.469901 16825 ts_manager.cc:194] Registered new tserver with Master: 1c5dbb0d234147e183aff1ac2e2b81d4 (127.6.215.194:44715)
W20260926 19:34:28.792265 17145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:28.792584 17145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:28.792732 17145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:28.799881 17145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:28.800016 17145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:28.800205 17145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:34:28.812481 17145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:37753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:28.814149 17145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:28.816404 17145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:28.829758 17150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:28.830482 17151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:28.831859 17145 server_base.cc:1034] running on GCE node
W20260926 19:34:28.832108 17154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:28.833063 17145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:28.834092 17145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:28.835251 17145 hybrid_clock.cc:648] HybridClock initialized: now 1790451268835189 us; error 51 us; skew 500 ppm
I20260926 19:34:28.835675 17145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:34:28.838608 17145 webserver.cc:460] Webserver started at http://127.6.215.195:35093/ using document root <none> and password file <none>
I20260926 19:34:28.839622 17145 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:28.839746 17145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:28.840179 17145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:28.844398 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "29609f13ed1a4725bf9a7766689bba0b"
format_stamp: "Formatted at 2026-09-26 19:34:28 on dist-test-slave-phbh"
server_key: "ff1f95480e75fe5bea7f5648fbb791c9"
server_key_iv: "257011b27b066b577a7b2b846829e491"
server_key_version: "encryptionkey@0"
I20260926 19:34:28.845546 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "29609f13ed1a4725bf9a7766689bba0b"
format_stamp: "Formatted at 2026-09-26 19:34:28 on dist-test-slave-phbh"
server_key: "ff1f95480e75fe5bea7f5648fbb791c9"
server_key_iv: "257011b27b066b577a7b2b846829e491"
server_key_version: "encryptionkey@0"
I20260926 19:34:28.850921 17145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:34:28.855610 17159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:28.857365 17145 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 19:34:28.857609 17145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "29609f13ed1a4725bf9a7766689bba0b"
format_stamp: "Formatted at 2026-09-26 19:34:28 on dist-test-slave-phbh"
server_key: "ff1f95480e75fe5bea7f5648fbb791c9"
server_key_iv: "257011b27b066b577a7b2b846829e491"
server_key_version: "encryptionkey@0"
I20260926 19:34:28.857942 17145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:28.865042 17145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:28.865816 17145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:28.866046 17145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:28.866914 17145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:28.868240 17145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:28.868324 17145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:28.868402 17145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:28.868455 17145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:28.900683 17145 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:43211
I20260926 19:34:28.900734 17271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:43211 every 8 connection(s)
I20260926 19:34:28.902048 17145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:34:28.905993 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 17145
I20260926 19:34:28.906176 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:34:28.906607 7007 external_mini_cluster.cc:1428] Setting key d535bf62245fd471c0557c62d19dbbe3
I20260926 19:34:28.910485 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:28.904485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:46931 (local address 127.6.215.254:37753)
0926 19:34:28.904666 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:28.904671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:28.906226 (+ 1555us) server_negotiation.cc:407] Connection header received
0926 19:34:28.908043 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:28.908048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:28.908112 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:28.908236 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:28.909754 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:28.909760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:28.909765 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:28.909783 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:28.909820 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:28.909831 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:28.909950 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:28.910162 (+ 212us) server_negotiation.cc:299] Negotiation successful
0926 19:34:28.910224 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:34:28.911381 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:28.905049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:37753 (local address 127.6.215.195:46931)
0926 19:34:28.906038 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:28.906101 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:28.907798 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:28.908452 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:28.908485 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:28.908614 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:28.909433 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:34:28.909439 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:28.909488 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:34:28.909593 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:28.909932 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:28.909942 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:28.909950 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 19:34:28.910226 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 19:34:28.910477 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":300,"threads_started":1}
I20260926 19:34:28.912621 17272 heartbeater.cc:347] Connected to a master server at 127.6.215.254:37753
I20260926 19:34:28.912930 17272 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:28.913477 17272 heartbeater.cc:511] Master 127.6.215.254:37753 requested a full tablet report, sending...
I20260926 19:34:28.914740 16825 ts_manager.cc:194] Registered new tserver with Master: 29609f13ed1a4725bf9a7766689bba0b (127.6.215.195:43211)
I20260926 19:34:28.921762 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 19:34:28.927737 17013 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:34:28.926172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.6.215.254:37753)
0926 19:34:28.926353 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:28.926357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:28.926895 (+ 538us) server_negotiation.cc:407] Connection header received
0926 19:34:28.927038 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:28.927042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:28.927092 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:28.927175 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:28.927451 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33676: BlockingRecv error: recv got EOF from 127.0.0.1:33676 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 19:34:28.928624 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16884
I20260926 19:34:28.932003 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:28.929593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.6.215.193:36957)
0926 19:34:28.930113 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:28.930120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:28.930154 (+ 34us) server_negotiation.cc:407] Connection header received
0926 19:34:28.930208 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:28.930214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:28.930431 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:28.930553 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:28.931461 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:28.931468 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:28.931480 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:28.931492 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:28.931535 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:28.931551 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:28.931654 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:28.931723 (+ 69us) server_negotiation.cc:299] Negotiation successful
0926 19:34:28.931801 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":255,"threads_started":1}
I20260926 19:34:28.933159 17006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:29.014355 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 16884
I20260926 19:34:29.030802 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 17015
I20260926 19:34:29.034184 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:29.031748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.6.215.194:44715)
0926 19:34:29.032258 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:29.032265 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:29.032292 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:34:29.032351 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:29.032358 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:29.032572 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:29.032700 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:29.033583 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:29.033590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:29.033601 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:29.033614 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:29.033656 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:29.033672 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:29.033773 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:29.033852 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 19:34:29.033929 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":252,"threads_started":1}
I20260926 19:34:29.035429 17137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:29.115793 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 17015
I20260926 19:34:29.131479 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 17145
I20260926 19:34:29.135476 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:29.132428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.6.215.195:43211)
0926 19:34:29.133097 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:29.133103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:29.133132 (+ 29us) server_negotiation.cc:407] Connection header received
0926 19:34:29.133190 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:29.133196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:29.133406 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:29.133579 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:29.134694 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:29.134700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:29.134712 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:34:29.134728 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:29.134765 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:34:29.134779 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:29.134965 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:29.135104 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 19:34:29.135210 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":358,"threads_started":1}
I20260926 19:34:29.136729 17267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:29.218075 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 17145
I20260926 19:34:29.233922 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 16794
I20260926 19:34:29.235165 16855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:29.336637 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 16794
2026-09-26T19:34:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2328 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:34:29 dist-test-slave-phbh krb5kdc[17290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:34:29 dist-test-slave-phbh krb5kdc[17290](info): set up 2 sockets
Sep 26 19:34:29 dist-test-slave-phbh krb5kdc[17290](info): commencing operation
krb5kdc: starting...
W20260926 19:34:30.441756 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:34:30 dist-test-slave-phbh krb5kdc[17290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:34:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:34:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:34:30.677012 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:38387
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:38387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:30.998126 17303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:30.998412 17303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:30.998634 17303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:31.005908 17303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:34:31.006068 17303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:31.006170 17303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:31.006278 17303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:34:31.006354 17303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:34:31.016388 17303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:38387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:38387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:31.017835 17303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:31.019184 17303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:31.030526 17308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:31.030622 17311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:31.030654 17309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:31.031548 17303 server_base.cc:1034] running on GCE node
I20260926 19:34:31.032292 17303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:31.033764 17303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:31.035005 17303 hybrid_clock.cc:648] HybridClock initialized: now 1790451271034949 us; error 62 us; skew 500 ppm
I20260926 19:34:31.035351 17303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:31 dist-test-slave-phbh krb5kdc[17290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451271, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:31.043483 17303 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:34:31.045068 17303 webserver.cc:460] Webserver started at http://127.6.215.254:33597/ using document root <none> and password file <none>
I20260926 19:34:31.045770 17303 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:31.045923 17303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:31.046238 17303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:31.049378 17303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "16acf2e4b91148b49f6f17b25458ade5"
format_stamp: "Formatted at 2026-09-26 19:34:31 on dist-test-slave-phbh"
server_key: "c9eea53c1432df90796c6440af44d349"
server_key_iv: "d9aad43758aba0620a21f30de3beaabe"
server_key_version: "encryptionkey@0"
I20260926 19:34:31.050168 17303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "16acf2e4b91148b49f6f17b25458ade5"
format_stamp: "Formatted at 2026-09-26 19:34:31 on dist-test-slave-phbh"
server_key: "c9eea53c1432df90796c6440af44d349"
server_key_iv: "d9aad43758aba0620a21f30de3beaabe"
server_key_version: "encryptionkey@0"
I20260926 19:34:31.055576 17303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:34:31.059229 17318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:31.060608 17303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 19:34:31.060851 17303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "16acf2e4b91148b49f6f17b25458ade5"
format_stamp: "Formatted at 2026-09-26 19:34:31 on dist-test-slave-phbh"
server_key: "c9eea53c1432df90796c6440af44d349"
server_key_iv: "d9aad43758aba0620a21f30de3beaabe"
server_key_version: "encryptionkey@0"
I20260926 19:34:31.061069 17303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:31.075484 17303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:31.079843 17303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:31.080130 17303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:31.098559 17303 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:38387
I20260926 19:34:31.098594 17369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:38387 every 8 connection(s)
I20260926 19:34:31.100020 17303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:34:31.103654 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:31.110080 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 17303
I20260926 19:34:31.110320 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:34:31.111011 7007 external_mini_cluster.cc:1428] Setting key e3c48f163e18f5ba53464e6a856ef963
I20260926 19:34:31.114112 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5: Bootstrap starting.
I20260926 19:34:31.118046 17370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:31.119199 17370 log.cc:824] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:31.122102 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5: No bootstrap required, opened a new log
Sep 26 19:34:31 dist-test-slave-phbh krb5kdc[17290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:31.131455 17370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16acf2e4b91148b49f6f17b25458ade5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38387 } }
I20260926 19:34:31.131922 17370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:31.132018 17370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16acf2e4b91148b49f6f17b25458ade5, State: Initialized, Role: FOLLOWER
I20260926 19:34:31.132645 17370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16acf2e4b91148b49f6f17b25458ade5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38387 } }
I20260926 19:34:31.132910 17370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:34:31.133040 17370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:34:31.133242 17370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:31.135661 17370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16acf2e4b91148b49f6f17b25458ade5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38387 } }
I20260926 19:34:31.136096 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:31.113131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.6.215.254:38387)
0926 19:34:31.113945 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:31.113961 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:31.114010 (+ 49us) server_negotiation.cc:407] Connection header received
0926 19:34:31.116430 (+ 2420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:31.116457 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:31.116950 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:31.117384 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:31.118793 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:31.120522 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:31.121833 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:31.122896 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:31.123070 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:31.127517 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:31.127528 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:31.127548 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:31.127592 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:31.131597 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:31.132496 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:31.132503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:31.132509 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:31.132676 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:31.133145 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:31.133149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:31.133151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:31.133785 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:31.134140 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:31.134651 (+ 511us) server_negotiation.cc:299] Negotiation successful
0926 19:34:31.135036 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":329,"threads_started":1}
I20260926 19:34:31.136194 17370 leader_election.cc:302] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16acf2e4b91148b49f6f17b25458ade5; no voters:
I20260926 19:34:31.136957 17370 leader_election.cc:288] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:34:31.137068 17375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:31.137738 17375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [term 1 LEADER]: Becoming Leader. State: Replica: 16acf2e4b91148b49f6f17b25458ade5, State: Running, Role: LEADER
I20260926 19:34:31.138505 17375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16acf2e4b91148b49f6f17b25458ade5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38387 } }
I20260926 19:34:31.140682 17370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:34:31.141554 17376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16acf2e4b91148b49f6f17b25458ade5. Latest consensus state: current_term: 1 leader_uuid: "16acf2e4b91148b49f6f17b25458ade5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16acf2e4b91148b49f6f17b25458ade5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38387 } } }
I20260926 19:34:31.141657 17377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16acf2e4b91148b49f6f17b25458ade5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16acf2e4b91148b49f6f17b25458ade5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 38387 } } }
I20260926 19:34:31.141912 17376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:31.141999 17377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:34:31.142910 17380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:34:31.149376 17380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:34:31.159183 17380 catalog_manager.cc:1347] Generated new cluster ID: 4a2610479cb94e2caee0689012873291
I20260926 19:34:31.159308 17380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:34:31.179083 17380 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:34:31.180356 17380 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:34:31.187649 17380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5: Generated new TSK 0
I20260926 19:34:31.188437 17380 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:34:31.248720 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38387
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:31.585690 17396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:31.586043 17396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:31.586220 17396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:31.593621 17396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:31.593783 17396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:31.593963 17396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:34:31.604480 17396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:31.605867 17396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:31.607188 17396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:31.620708 17401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:31.620741 17402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:31.620846 17404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:31.621879 17396 server_base.cc:1034] running on GCE node
I20260926 19:34:31.622516 17396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:31.623559 17396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:31.624761 17396 hybrid_clock.cc:648] HybridClock initialized: now 1790451271624697 us; error 56 us; skew 500 ppm
I20260926 19:34:31.625110 17396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:31 dist-test-slave-phbh krb5kdc[17290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451271, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:31.628499 17396 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:34:31.630017 17396 webserver.cc:460] Webserver started at http://127.6.215.193:40665/ using document root <none> and password file <none>
I20260926 19:34:31.630700 17396 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:31.630844 17396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:31.631143 17396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:31.634007 17396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "a2b4038a370c48219f0201ac75015e43"
format_stamp: "Formatted at 2026-09-26 19:34:31 on dist-test-slave-phbh"
server_key: "67e53e908fbedf7d4af4ec07dacd28b8"
server_key_iv: "fe6ef1b6241a6e4d4343a82820104aa3"
server_key_version: "encryptionkey@0"
I20260926 19:34:31.634759 17396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "a2b4038a370c48219f0201ac75015e43"
format_stamp: "Formatted at 2026-09-26 19:34:31 on dist-test-slave-phbh"
server_key: "67e53e908fbedf7d4af4ec07dacd28b8"
server_key_iv: "fe6ef1b6241a6e4d4343a82820104aa3"
server_key_version: "encryptionkey@0"
I20260926 19:34:31.640158 17396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260926 19:34:31.643610 17411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:31.644999 17396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:34:31.645164 17396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "a2b4038a370c48219f0201ac75015e43"
format_stamp: "Formatted at 2026-09-26 19:34:31 on dist-test-slave-phbh"
server_key: "67e53e908fbedf7d4af4ec07dacd28b8"
server_key_iv: "fe6ef1b6241a6e4d4343a82820104aa3"
server_key_version: "encryptionkey@0"
I20260926 19:34:31.645366 17396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:31.675561 17396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:31.679983 17396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:31.680294 17396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:31.681262 17396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:31.682617 17396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:31.682734 17396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:31.682883 17396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:31.682956 17396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:31.716943 17396 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:35595
I20260926 19:34:31.717025 17523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:35595 every 8 connection(s)
I20260926 19:34:31.718250 17396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:34:31.725132 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 17396
I20260926 19:34:31.725358 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:34:31.726047 7007 external_mini_cluster.cc:1428] Setting key 4dcf14baa594f55760dec62df0e70292
Sep 26 19:34:31 dist-test-slave-phbh krb5kdc[17290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451271, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:31.740060 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:31.720824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:41173 (local address 127.6.215.254:38387)
0926 19:34:31.721269 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:31.721275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:31.722262 (+ 987us) server_negotiation.cc:407] Connection header received
0926 19:34:31.724633 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:31.724639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:31.724734 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:31.724900 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:31.726452 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:31.727243 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:31.729098 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:31.729884 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:31.730034 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:31.733381 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:31.733390 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:31.733398 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:31.733444 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:31.736493 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:31.737600 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:31.737604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:31.737607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:31.737743 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:31.738243 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:31.738247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:31.738249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:31.738537 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:31.738650 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:31.739628 (+ 978us) server_negotiation.cc:299] Negotiation successful
0926 19:34:31.739801 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":213,"threads_started":1}
I20260926 19:34:31.741567 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:31.721448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:38387 (local address 127.6.215.193:41173)
0926 19:34:31.722104 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:31.722143 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:31.724311 (+ 2168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:31.725080 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:31.725088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:31.725683 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:31.726280 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:31.726298 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:31.727420 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:31.727426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:31.728953 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:31.728965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:31.730102 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:31.730107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:31.730303 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:31.730976 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:31.730998 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:31.733159 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:31.736674 (+ 3515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:31.736686 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:31.736707 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:31.737356 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:31.737855 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:31.737863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:31.737869 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:31.738100 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:31.738682 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:31.738693 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:31.739155 (+ 462us) client_negotiation.cc:769] Sending connection context
0926 19:34:31.739452 (+ 297us) client_negotiation.cc:240] Negotiation successful
0926 19:34:31.740035 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":276,"threads_started":1}
I20260926 19:34:31.743320 17524 heartbeater.cc:347] Connected to a master server at 127.6.215.254:38387
I20260926 19:34:31.743657 17524 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:31.744644 17524 heartbeater.cc:511] Master 127.6.215.254:38387 requested a full tablet report, sending...
I20260926 19:34:31.747288 17335 ts_manager.cc:194] Registered new tserver with Master: a2b4038a370c48219f0201ac75015e43 (127.6.215.193:35595)
I20260926 19:34:31.749612 17335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:41173
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:34:31.790439 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38387
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:32.122401 17530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:32.122730 17530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:32.122896 17530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:32.130311 17530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:32.130467 17530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:32.130652 17530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:34:32.140386 17530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:32.141876 17530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:32.143246 17530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:32.156314 17535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:32.156378 17536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:32.156352 17538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:32.157328 17530 server_base.cc:1034] running on GCE node
I20260926 19:34:32.157933 17530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:32.158813 17530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:32.160054 17530 hybrid_clock.cc:648] HybridClock initialized: now 1790451272160007 us; error 65 us; skew 500 ppm
I20260926 19:34:32.160365 17530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:32 dist-test-slave-phbh krb5kdc[17290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:32.163460 17530 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:34:32.164942 17530 webserver.cc:460] Webserver started at http://127.6.215.194:37129/ using document root <none> and password file <none>
I20260926 19:34:32.165675 17530 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:32.165771 17530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:32.166103 17530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:32.168967 17530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "cfca8163eb82484bb50b98ce19ee8fe3"
format_stamp: "Formatted at 2026-09-26 19:34:32 on dist-test-slave-phbh"
server_key: "f6a13c06b56874e1ca5d8e586b75a4ae"
server_key_iv: "56ba6ced863a0afa2ea280a239f43941"
server_key_version: "encryptionkey@0"
I20260926 19:34:32.169737 17530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "cfca8163eb82484bb50b98ce19ee8fe3"
format_stamp: "Formatted at 2026-09-26 19:34:32 on dist-test-slave-phbh"
server_key: "f6a13c06b56874e1ca5d8e586b75a4ae"
server_key_iv: "56ba6ced863a0afa2ea280a239f43941"
server_key_version: "encryptionkey@0"
I20260926 19:34:32.174659 17530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:34:32.178084 17545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:32.179680 17530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:34:32.179847 17530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "cfca8163eb82484bb50b98ce19ee8fe3"
format_stamp: "Formatted at 2026-09-26 19:34:32 on dist-test-slave-phbh"
server_key: "f6a13c06b56874e1ca5d8e586b75a4ae"
server_key_iv: "56ba6ced863a0afa2ea280a239f43941"
server_key_version: "encryptionkey@0"
I20260926 19:34:32.180078 17530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:32.197863 17530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:32.202075 17530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:32.202307 17530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:32.203188 17530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:32.204491 17530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:32.204560 17530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:32.204653 17530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:32.204699 17530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:32.236315 17530 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:39387
I20260926 19:34:32.236413 17657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:39387 every 8 connection(s)
I20260926 19:34:32.237669 17530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:34:32.246693 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 17530
I20260926 19:34:32.246886 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:34:32.247318 7007 external_mini_cluster.cc:1428] Setting key dc8b162c9f425ecbe077a472415f8e84
Sep 26 19:34:32 dist-test-slave-phbh krb5kdc[17290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:32.259379 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.240242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:60979 (local address 127.6.215.254:38387)
0926 19:34:32.240443 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:32.240448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:32.241775 (+ 1327us) server_negotiation.cc:407] Connection header received
0926 19:34:32.243537 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:32.243543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:32.243638 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:32.243750 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:32.245284 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.245971 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.247668 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.248152 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.248286 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.252882 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:32.252887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:32.252890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:32.252921 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:32.255855 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:32.256843 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:32.256852 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:32.256857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:32.257035 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:32.257539 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:32.257546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:32.257549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:32.257921 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:32.258033 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:32.258977 (+ 944us) server_negotiation.cc:299] Negotiation successful
0926 19:34:32.259146 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 19:34:32.261052 17660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.240880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:38387 (local address 127.6.215.194:60979)
0926 19:34:32.241610 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:32.241670 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:32.243273 (+ 1603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:32.243985 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:32.243993 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:32.244489 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:32.245110 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.245134 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.246124 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.246133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.247511 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.247522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.248563 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.248570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.248879 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.249648 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:32.249674 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:32.252491 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:32.256027 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:32.256039 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:32.256062 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:32.256674 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:32.257155 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:32.257158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:32.257161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:32.257332 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:32.258053 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:32.258063 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:32.258560 (+ 497us) client_negotiation.cc:769] Sending connection context
0926 19:34:32.258881 (+ 321us) client_negotiation.cc:240] Negotiation successful
0926 19:34:32.259490 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":257,"threads_started":1}
I20260926 19:34:32.262578 17658 heartbeater.cc:347] Connected to a master server at 127.6.215.254:38387
I20260926 19:34:32.262897 17658 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:32.263650 17658 heartbeater.cc:511] Master 127.6.215.254:38387 requested a full tablet report, sending...
I20260926 19:34:32.265260 17335 ts_manager.cc:194] Registered new tserver with Master: cfca8163eb82484bb50b98ce19ee8fe3 (127.6.215.194:39387)
I20260926 19:34:32.266155 17335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:60979
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:34:32.300696 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38387
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:34:32.629336 17663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:34:32.629681 17663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:34:32.629850 17663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:34:32.637218 17663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:34:32.637416 17663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:34:32.637598 17663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:34:32.647372 17663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:38387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:34:32.648835 17663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:34:32.650328 17663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:34:32.663620 17668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:32.663677 17669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:34:32.663599 17671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:34:32.664450 17663 server_base.cc:1034] running on GCE node
I20260926 19:34:32.665135 17663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:34:32.666141 17663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:34:32.667383 17663 hybrid_clock.cc:648] HybridClock initialized: now 1790451272667320 us; error 55 us; skew 500 ppm
I20260926 19:34:32.667732 17663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:34:32 dist-test-slave-phbh krb5kdc[17290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:34:32.670812 17663 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:34:32.672238 17663 webserver.cc:460] Webserver started at http://127.6.215.195:35073/ using document root <none> and password file <none>
I20260926 19:34:32.672984 17663 fs_manager.cc:359] Metadata directory not provided
I20260926 19:34:32.673130 17663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:34:32.673418 17663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:34:32.676265 17663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "c6dce014b7564563af8cf4a27bda17e4"
format_stamp: "Formatted at 2026-09-26 19:34:32 on dist-test-slave-phbh"
server_key: "46c7df772c04479892a3b24b74edc408"
server_key_iv: "f83b8755650dab0e12428c4a21107c20"
server_key_version: "encryptionkey@0"
I20260926 19:34:32.677099 17663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "c6dce014b7564563af8cf4a27bda17e4"
format_stamp: "Formatted at 2026-09-26 19:34:32 on dist-test-slave-phbh"
server_key: "46c7df772c04479892a3b24b74edc408"
server_key_iv: "f83b8755650dab0e12428c4a21107c20"
server_key_version: "encryptionkey@0"
I20260926 19:34:32.682228 17663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 19:34:32.685920 17678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:32.687170 17663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:34:32.687354 17663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "c6dce014b7564563af8cf4a27bda17e4"
format_stamp: "Formatted at 2026-09-26 19:34:32 on dist-test-slave-phbh"
server_key: "46c7df772c04479892a3b24b74edc408"
server_key_iv: "f83b8755650dab0e12428c4a21107c20"
server_key_version: "encryptionkey@0"
I20260926 19:34:32.687644 17663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:34:32.707258 17663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:34:32.711347 17663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:34:32.711663 17663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:34:32.712621 17663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:34:32.713980 17663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:34:32.714102 17663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:32.714243 17663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:34:32.714315 17663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:34:32.751579 17663 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:33633
I20260926 19:34:32.751614 17790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:33633 every 8 connection(s)
I20260926 19:34:32.752857 17663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:34:32.753201 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 17663
I20260926 19:34:32.753412 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:34:32.753731 17524 heartbeater.cc:503] Master 127.6.215.254:38387 was elected leader, sending a full tablet report...
I20260926 19:34:32.753908 7007 external_mini_cluster.cc:1428] Setting key 6cedf55d062e6db2b88998615ec7ee22
Sep 26 19:34:32 dist-test-slave-phbh krb5kdc[17290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451272, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:34:32.772645 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.756485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:38537 (local address 127.6.215.254:38387)
0926 19:34:32.756694 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:32.756700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:32.757432 (+ 732us) server_negotiation.cc:407] Connection header received
0926 19:34:32.759081 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:32.759085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:32.759144 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:32.759229 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:32.760584 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.761374 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.762788 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.763170 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.763261 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.766703 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:32.766707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:32.766711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:32.766739 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:32.769778 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:32.770466 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:32.770472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:32.770476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:32.770600 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:32.771010 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:32.771017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:32.771020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:32.771286 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:32.771435 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:32.772078 (+ 643us) server_negotiation.cc:299] Negotiation successful
0926 19:34:32.772278 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:34:32.773587 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.756487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:38387 (local address 127.6.215.195:38537)
0926 19:34:32.757276 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:32.757312 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:32.758820 (+ 1508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:32.759429 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:32.759436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:32.759896 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:34:32.760436 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.760452 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.761492 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.761495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.762668 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.762678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.763349 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.763353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.763537 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.764204 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:34:32.764229 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:34:32.766536 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:34:32.769917 (+ 3381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:32.769925 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:32.769941 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:32.770344 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:32.770715 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:34:32.770721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:34:32.770724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:34:32.770901 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:34:32.771407 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:34:32.771415 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:34:32.771730 (+ 315us) client_negotiation.cc:769] Sending connection context
0926 19:34:32.771990 (+ 260us) client_negotiation.cc:240] Negotiation successful
0926 19:34:32.772495 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":328,"threads_started":1}
I20260926 19:34:32.774863 17791 heartbeater.cc:347] Connected to a master server at 127.6.215.254:38387
I20260926 19:34:32.775120 17791 heartbeater.cc:464] Registering TS with master...
I20260926 19:34:32.775769 17791 heartbeater.cc:511] Master 127.6.215.254:38387 requested a full tablet report, sending...
I20260926 19:34:32.777064 17335 ts_manager.cc:194] Registered new tserver with Master: c6dce014b7564563af8cf4a27bda17e4 (127.6.215.195:33633)
I20260926 19:34:32.777810 17335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:38537
I20260926 19:34:32.781695 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:34:32.800597 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.786633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.6.215.254:38387)
0926 19:34:32.786841 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:32.786848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:32.787571 (+ 723us) server_negotiation.cc:407] Connection header received
0926 19:34:32.787791 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:32.787797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:32.787887 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:32.788011 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:32.789577 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.790360 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.791710 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.792174 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.792318 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.794904 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:32.794911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:32.794916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:32.794981 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:32.797675 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:32.798395 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:32.798403 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:32.798407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:32.798559 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:32.798985 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:32.798991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:32.798995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:32.799372 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:32.799686 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:32.800037 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 19:34:32.800258 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 19:34:32.804642 17335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:34:32.811353 17335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:34:32.838800 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.824863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:33633 (local address 127.0.0.1:37894)
0926 19:34:32.825302 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:32.825321 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:32.825429 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:32.826591 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:32.826598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:32.826648 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:32.827189 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.827205 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.830585 (+ 3380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.830598 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.833618 (+ 3020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.833639 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.837340 (+ 3701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.837349 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.837673 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.837759 (+ 86us) client_negotiation.cc:769] Sending connection context
0926 19:34:32.837902 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 19:34:32.838155 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":32,"thread_start_us":615,"threads_started":1}
I20260926 19:34:32.839691 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.823262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45670)
0926 19:34:32.824422 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:32.824458 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:32.824662 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:32.825441 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:32.825460 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:32.825496 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:32.825944 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.825959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.829363 (+ 3404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.829372 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.834444 (+ 5072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.834459 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.838422 (+ 3963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.838433 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.838745 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.838809 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 19:34:32.838892 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 19:34:32.839048 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"mutex_wait_us":1981,"thread_start_us":807,"threads_started":1}
I20260926 19:34:32.840553 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.825254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.6.215.195:33633)
0926 19:34:32.825948 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:32.825957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:32.825987 (+ 30us) server_negotiation.cc:407] Connection header received
0926 19:34:32.826073 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:32.826085 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:32.826349 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:32.827442 (+ 1093us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:32.827777 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.830395 (+ 2618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.834687 (+ 4292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.837061 (+ 2374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.839447 (+ 2386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.839788 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:32.839877 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 19:34:32.840108 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":660,"threads_started":1}
I20260926 19:34:32.842147 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.823400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45670 (local address 127.6.215.193:35595)
0926 19:34:32.824058 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:32.824067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:32.824599 (+ 532us) server_negotiation.cc:407] Connection header received
0926 19:34:32.824922 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:32.824933 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:32.825214 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:32.825429 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:32.826170 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.829151 (+ 2981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.835741 (+ 6590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.838059 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.841312 (+ 3253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.841561 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:32.841618 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 19:34:32.841741 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":317,"threads_started":1}
I20260926 19:34:32.845233 17726 tablet_service.cc:1467] Processing CreateTablet for tablet 716f4b30b4374e95afc978b7ccaa8cce (DEFAULT_TABLE table=test-table [id=672fc609be7d463da2751bf309dd4fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:32.845309 17459 tablet_service.cc:1467] Processing CreateTablet for tablet 716f4b30b4374e95afc978b7ccaa8cce (DEFAULT_TABLE table=test-table [id=672fc609be7d463da2751bf309dd4fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:32.846638 17726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 716f4b30b4374e95afc978b7ccaa8cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:32.846642 17459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 716f4b30b4374e95afc978b7ccaa8cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:32.849339 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.826071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.6.215.194:39387)
0926 19:34:32.841496 (+ 15425us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:32.841508 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:32.841538 (+ 30us) server_negotiation.cc:407] Connection header received
0926 19:34:32.841645 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:32.841653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:32.841935 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:32.842167 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:32.843209 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.844681 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.846885 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.848291 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.848435 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.848664 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:32.848894 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 19:34:32.849052 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15255,"thread_start_us":15157,"threads_started":1}
I20260926 19:34:32.849372 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:32.825789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:39387 (local address 127.0.0.1:35576)
0926 19:34:32.839098 (+ 13309us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:32.839119 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:32.839246 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:32.842145 (+ 2899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:32.842150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:32.842180 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:32.842361 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.842375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.844859 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.844865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.846678 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:32.846709 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:32.848482 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:32.848488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:32.848691 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:32.848723 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 19:34:32.848845 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 19:34:32.848980 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13181,"thread_start_us":15148,"threads_started":1}
I20260926 19:34:32.853191 17593 tablet_service.cc:1467] Processing CreateTablet for tablet 716f4b30b4374e95afc978b7ccaa8cce (DEFAULT_TABLE table=test-table [id=672fc609be7d463da2751bf309dd4fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:34:32.854513 17593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 716f4b30b4374e95afc978b7ccaa8cce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:34:32.858479 17809 tablet_bootstrap.cc:492] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Bootstrap starting.
I20260926 19:34:32.860428 17810 tablet_bootstrap.cc:492] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: Bootstrap starting.
I20260926 19:34:32.862561 17809 tablet_bootstrap.cc:654] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:32.863679 17810 tablet_bootstrap.cc:654] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:32.863886 17809 log.cc:824] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:32.864900 17810 log.cc:824] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:32.866986 17809 tablet_bootstrap.cc:492] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: No bootstrap required, opened a new log
I20260926 19:34:32.867331 17809 ts_tablet_manager.cc:1397] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 19:34:32.867391 17813 tablet_bootstrap.cc:492] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Bootstrap starting.
I20260926 19:34:32.867934 17810 tablet_bootstrap.cc:492] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: No bootstrap required, opened a new log
I20260926 19:34:32.868192 17810 ts_tablet_manager.cc:1397] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 19:34:32.870249 17813 tablet_bootstrap.cc:654] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Neither blocks nor log segments found. Creating new log.
I20260926 19:34:32.871228 17813 log.cc:824] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Log is configured to *not* fsync() on all Append() calls
I20260926 19:34:32.875051 17813 tablet_bootstrap.cc:492] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: No bootstrap required, opened a new log
I20260926 19:34:32.875337 17813 ts_tablet_manager.cc:1397] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260926 19:34:32.878393 17809 raft_consensus.cc:348] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:32.878880 17809 raft_consensus.cc:374] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:32.878679 17810 raft_consensus.cc:348] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:32.878975 17809 raft_consensus.cc:729] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6dce014b7564563af8cf4a27bda17e4, State: Initialized, Role: FOLLOWER
I20260926 19:34:32.879086 17810 raft_consensus.cc:374] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:32.879185 17810 raft_consensus.cc:729] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b4038a370c48219f0201ac75015e43, State: Initialized, Role: FOLLOWER
I20260926 19:34:32.879628 17809 consensus_queue.cc:260] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:32.879777 17810 consensus_queue.cc:260] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:32.880813 17791 heartbeater.cc:503] Master 127.6.215.254:38387 was elected leader, sending a full tablet report...
I20260926 19:34:32.881481 17809 ts_tablet_manager.cc:1428] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 19:34:32.882990 17810 ts_tablet_manager.cc:1428] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260926 19:34:32.883771 17813 raft_consensus.cc:348] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:32.884235 17813 raft_consensus.cc:374] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:34:32.884341 17813 raft_consensus.cc:729] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfca8163eb82484bb50b98ce19ee8fe3, State: Initialized, Role: FOLLOWER
I20260926 19:34:32.884976 17813 consensus_queue.cc:260] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:32.886574 17658 heartbeater.cc:503] Master 127.6.215.254:38387 was elected leader, sending a full tablet report...
I20260926 19:34:32.887271 17813 ts_tablet_manager.cc:1428] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260926 19:34:32.970925 17525 tablet.cc:2369] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:34:32.989856 17659 tablet.cc:2369] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:34:33.006731 17792 tablet.cc:2369] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:34:33.178645 17815 raft_consensus.cc:482] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:34:33.179020 17815 raft_consensus.cc:504] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:33.180944 17815 leader_election.cc:288] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b4038a370c48219f0201ac75015e43 (127.6.215.193:35595), cfca8163eb82484bb50b98ce19ee8fe3 (127.6.215.194:39387)
I20260926 19:34:33.185848 17817 raft_consensus.cc:482] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:34:33.186527 17817 raft_consensus.cc:504] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:33.188265 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.181183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.6.215.195:54635)
0926 19:34:33.181387 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.181406 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:33.181620 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:33.181960 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:33.181966 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:33.181995 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:33.182255 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.182271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.183745 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.183753 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.185815 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.185834 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.187511 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.187517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.187704 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.187741 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 19:34:33.187833 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 19:34:33.187970 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":1}
I20260926 19:34:33.188313 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.181405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54635 (local address 127.6.215.193:35595)
0926 19:34:33.181630 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.181636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.181661 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:34:33.181741 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.181747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.181822 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.181946 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:33.182454 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.183554 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.186026 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.187216 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.187357 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.187432 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.187902 (+ 470us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.188071 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 19:34:33.189182 17817 leader_election.cc:288] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b4038a370c48219f0201ac75015e43 (127.6.215.193:35595), c6dce014b7564563af8cf4a27bda17e4 (127.6.215.195:33633)
I20260926 19:34:33.189728 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.181934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:37755 (local address 127.6.215.194:39387)
0926 19:34:33.182189 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.182198 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.182700 (+ 502us) server_negotiation.cc:407] Connection header received
0926 19:34:33.182804 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.182809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.182887 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.182976 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:33.183695 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.184875 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.187276 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.188599 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.188716 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.188801 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.189339 (+ 538us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.189473 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 19:34:33.190364 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.181705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:39387 (local address 127.6.215.195:37755)
0926 19:34:33.182592 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.182606 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:33.182703 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:33.183086 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:33.183090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:33.183120 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:33.183521 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.183536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.185043 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.185048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.187092 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.187107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.188952 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.188956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.189145 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.189175 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 19:34:33.189884 (+ 709us) client_negotiation.cc:240] Negotiation successful
0926 19:34:33.190026 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":539,"threads_started":1}
I20260926 19:34:33.191423 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "716f4b30b4374e95afc978b7ccaa8cce" candidate_uuid: "c6dce014b7564563af8cf4a27bda17e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" is_pre_election: true
I20260926 19:34:33.191526 17479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "716f4b30b4374e95afc978b7ccaa8cce" candidate_uuid: "c6dce014b7564563af8cf4a27bda17e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b4038a370c48219f0201ac75015e43" is_pre_election: true
I20260926 19:34:33.192026 17613 raft_consensus.cc:2454] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6dce014b7564563af8cf4a27bda17e4 in term 0.
I20260926 19:34:33.192108 17479 raft_consensus.cc:2454] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6dce014b7564563af8cf4a27bda17e4 in term 0.
I20260926 19:34:33.192904 17679 leader_election.cc:302] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2b4038a370c48219f0201ac75015e43, c6dce014b7564563af8cf4a27bda17e4; no voters:
I20260926 19:34:33.193326 17815 raft_consensus.cc:2781] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:34:33.193504 17815 raft_consensus.cc:482] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:34:33.193632 17815 raft_consensus.cc:3037] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:33.197054 17815 raft_consensus.cc:504] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:33.197857 17815 leader_election.cc:288] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [CANDIDATE]: Term 1 election: Requested vote from peers a2b4038a370c48219f0201ac75015e43 (127.6.215.193:35595), cfca8163eb82484bb50b98ce19ee8fe3 (127.6.215.194:39387)
I20260926 19:34:33.198346 17613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "716f4b30b4374e95afc978b7ccaa8cce" candidate_uuid: "c6dce014b7564563af8cf4a27bda17e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfca8163eb82484bb50b98ce19ee8fe3"
I20260926 19:34:33.198622 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.191147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:53089 (local address 127.6.215.195:33633)
0926 19:34:33.192341 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.192348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.192365 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:34:33.192414 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.192418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.192506 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.192592 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:33.193476 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.194345 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.196132 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.197431 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.197587 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.197660 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.198200 (+ 540us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.198354 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055}
I20260926 19:34:33.198568 17613 raft_consensus.cc:3037] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:33.199086 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.189495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.6.215.194:60147)
0926 19:34:33.190270 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.190302 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:33.190438 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:33.191755 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:33.191759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:33.191786 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:33.192220 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.192232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.193750 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.193756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.196206 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.196220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.198133 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.198142 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.198324 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.198365 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 19:34:33.198518 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 19:34:33.198656 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":1,"thread_start_us":807,"threads_started":1}
I20260926 19:34:33.199041 17479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "716f4b30b4374e95afc978b7ccaa8cce" candidate_uuid: "c6dce014b7564563af8cf4a27bda17e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b4038a370c48219f0201ac75015e43"
I20260926 19:34:33.199291 17479 raft_consensus.cc:3037] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:34:33.199774 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.190765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:33633 (local address 127.6.215.194:53089)
0926 19:34:33.191748 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.191906 (+ 158us) client_negotiation.cc:174] Beginning negotiation
0926 19:34:33.192021 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:34:33.192945 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:34:33.192950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:34:33.192973 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:34:33.193310 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.193320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.194500 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.194504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.195946 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:34:33.195957 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.197680 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.197687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:34:33.197903 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.198015 (+ 112us) client_negotiation.cc:769] Sending connection context
0926 19:34:33.199102 (+ 1087us) client_negotiation.cc:240] Negotiation successful
0926 19:34:33.199461 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":32,"thread_start_us":2005,"threads_started":1}
I20260926 19:34:33.200510 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.190032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:60147 (local address 127.6.215.193:35595)
0926 19:34:33.191118 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.191124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.191146 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:34:33.191200 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.191206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.191295 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.191400 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:34:33.192401 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.193630 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.196385 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.197719 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.197887 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.197968 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.198582 (+ 614us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.198857 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":1137,"threads_started":1}
I20260926 19:34:33.200654 17746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "716f4b30b4374e95afc978b7ccaa8cce" candidate_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6dce014b7564563af8cf4a27bda17e4" is_pre_election: true
I20260926 19:34:33.201153 17746 raft_consensus.cc:2379] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cfca8163eb82484bb50b98ce19ee8fe3 in current term 1: Already voted for candidate c6dce014b7564563af8cf4a27bda17e4 in this term.
I20260926 19:34:33.201227 17478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "716f4b30b4374e95afc978b7ccaa8cce" candidate_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b4038a370c48219f0201ac75015e43" is_pre_election: true
I20260926 19:34:33.202268 17613 raft_consensus.cc:2454] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6dce014b7564563af8cf4a27bda17e4 in term 1.
I20260926 19:34:33.202293 17546 leader_election.cc:302] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfca8163eb82484bb50b98ce19ee8fe3; no voters: a2b4038a370c48219f0201ac75015e43, c6dce014b7564563af8cf4a27bda17e4
I20260926 19:34:33.202687 17679 leader_election.cc:302] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6dce014b7564563af8cf4a27bda17e4, cfca8163eb82484bb50b98ce19ee8fe3; no voters:
I20260926 19:34:33.202740 17817 raft_consensus.cc:2726] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:34:33.202962 17479 raft_consensus.cc:2454] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6dce014b7564563af8cf4a27bda17e4 in term 1.
I20260926 19:34:33.203030 17815 raft_consensus.cc:2781] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:34:33.203570 17815 raft_consensus.cc:686] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 1 LEADER]: Becoming Leader. State: Replica: c6dce014b7564563af8cf4a27bda17e4, State: Running, Role: LEADER
I20260926 19:34:33.204089 17815 consensus_queue.cc:237] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } }
I20260926 19:34:33.208536 17333 catalog_manager.cc:5495] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6dce014b7564563af8cf4a27bda17e4 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "c6dce014b7564563af8cf4a27bda17e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6dce014b7564563af8cf4a27bda17e4" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 } health_report { overall_health: UNKNOWN } } }
I20260926 19:34:33.290721 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.285509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37904 (local address 127.6.215.195:33633)
0926 19:34:33.285706 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.285710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.285729 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:34:33.285947 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.285952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.286027 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.286157 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:34:33.286512 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.287244 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.288575 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.289051 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.289161 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.289610 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:34:33.290101 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:34:33.290218 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.290307 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.290367 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 19:34:33.298936 17479 raft_consensus.cc:1261] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 1 FOLLOWER]: Refusing update from remote peer c6dce014b7564563af8cf4a27bda17e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:33.298982 17613 raft_consensus.cc:1261] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 1 FOLLOWER]: Refusing update from remote peer c6dce014b7564563af8cf4a27bda17e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:34:33.299865 17822 consensus_queue.cc:1035] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b4038a370c48219f0201ac75015e43" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:33.300237 17815 consensus_queue.cc:1035] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfca8163eb82484bb50b98ce19ee8fe3" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:34:33.312537 17826 mvcc.cc:205] Tried to move back new op lower bound from 7333688415422173184 to 7333688415048253440. Current Snapshot: MvccSnapshot[applied={T|T < 7333688415422173184}]
I20260926 19:34:33.324564 17335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:33.324904 17335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:34:33.328284 17335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16acf2e4b91148b49f6f17b25458ade5: Sending DeleteTablet for 3 replicas of tablet 716f4b30b4374e95afc978b7ccaa8cce
I20260926 19:34:33.329463 17459 tablet_service.cc:1514] Processing DeleteTablet for tablet 716f4b30b4374e95afc978b7ccaa8cce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:33 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45670
I20260926 19:34:33.329577 17726 tablet_service.cc:1514] Processing DeleteTablet for tablet 716f4b30b4374e95afc978b7ccaa8cce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:33 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:37894
I20260926 19:34:33.329977 17593 tablet_service.cc:1514] Processing DeleteTablet for tablet 716f4b30b4374e95afc978b7ccaa8cce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:34:33 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:35576
I20260926 19:34:33.331506 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 17396
I20260926 19:34:33.340471 17836 tablet_replica.cc:330] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: stopping tablet replica
I20260926 19:34:33.340979 17836 raft_consensus.cc:2229] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:34:33.341766 17836 raft_consensus.cc:2258] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:33.346303 17836 ts_tablet_manager.cc:1905] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:33.348532 17837 tablet_replica.cc:330] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: stopping tablet replica
I20260926 19:34:33.349009 17837 raft_consensus.cc:2229] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:33.349589 17837 raft_consensus.cc:2258] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3 [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 26 19:34:33 dist-test-slave-phbh krb5kdc[17290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:34:33.348135 17835 tablet_replica.cc:330] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: stopping tablet replica
I20260926 19:34:33.351971 17837 ts_tablet_manager.cc:1905] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:33.353789 17835 raft_consensus.cc:2229] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:34:33.354667 17835 raft_consensus.cc:2258] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:34:33.355219 17836 ts_tablet_manager.cc:1918] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:34:33.355397 17836 log.cc:1196] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/716f4b30b4374e95afc978b7ccaa8cce
I20260926 19:34:33.356010 17836 ts_tablet_manager.cc:1939] T 716f4b30b4374e95afc978b7ccaa8cce P c6dce014b7564563af8cf4a27bda17e4: Deleting consensus metadata
I20260926 19:34:33.358055 17319 catalog_manager.cc:4841] TS c6dce014b7564563af8cf4a27bda17e4 (127.6.215.195:33633): tablet 716f4b30b4374e95afc978b7ccaa8cce (table test-table [id=672fc609be7d463da2751bf309dd4fd3]) successfully deleted
I20260926 19:34:33.359661 17837 ts_tablet_manager.cc:1918] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:34:33.359711 17835 ts_tablet_manager.cc:1905] T 716f4b30b4374e95afc978b7ccaa8cce P a2b4038a370c48219f0201ac75015e43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:34:33.359812 17837 log.cc:1196] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/716f4b30b4374e95afc978b7ccaa8cce
I20260926 19:34:33.360380 17837 ts_tablet_manager.cc:1939] T 716f4b30b4374e95afc978b7ccaa8cce P cfca8163eb82484bb50b98ce19ee8fe3: Deleting consensus metadata
I20260926 19:34:33.362182 17319 catalog_manager.cc:4841] TS cfca8163eb82484bb50b98ce19ee8fe3 (127.6.215.194:39387): tablet 716f4b30b4374e95afc978b7ccaa8cce (table test-table [id=672fc609be7d463da2751bf309dd4fd3]) successfully deleted
I20260926 19:34:33.362892 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.332456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.6.215.193:35595)
0926 19:34:33.332708 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.332717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.339468 (+ 6751us) server_negotiation.cc:407] Connection header received
0926 19:34:33.340368 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.340374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.340471 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.340549 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:33.342994 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.344364 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.346219 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.346803 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.346923 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.355121 (+ 8198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:33.355135 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:33.355152 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:33.355198 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:33.359266 (+ 4068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:33.360272 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:33.360282 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:33.360285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:33.360494 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:33.361039 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:33.361043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:33.361046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:33.361520 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:33.361834 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.362386 (+ 552us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.362639 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 19:34:33.364230 17519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:33.459380 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 17396
W20260926 19:34:33.484076 17319 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.215.193:35595 (error 108)
I20260926 19:34:33.484870 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 17530
I20260926 19:34:33.486876 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.485589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45688)
0926 19:34:33.486530 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.486659 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":727,"threads_started":1}
W20260926 19:34:33.487154 17319 catalog_manager.cc:4584] TS a2b4038a370c48219f0201ac75015e43 (127.6.215.193:35595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 716f4b30b4374e95afc978b7ccaa8cce: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Sep 26 19:34:33 dist-test-slave-phbh krb5kdc[17290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:34:33.500221 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.485626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.6.215.194:39387)
0926 19:34:33.485802 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.485808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.486006 (+ 198us) server_negotiation.cc:407] Connection header received
0926 19:34:33.486225 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.486231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.486319 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.486424 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:33.488004 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.488721 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.490014 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.490422 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.490553 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.494625 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:33.494638 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:33.494654 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:33.494686 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:33.497482 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:33.498187 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:33.498200 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:33.498203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:33.498331 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:33.498707 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:33.498717 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:33.498719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:33.499050 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:33.499340 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.499644 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.499877 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 19:34:33.501061 17653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:33.540834 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.540256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45696)
0926 19:34:33.540510 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.540622 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260926 19:34:33.542084 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.541626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45700)
0926 19:34:33.541801 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.541922 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 19:34:33.594357 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 17530
I20260926 19:34:33.602820 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.602387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45714)
0926 19:34:33.602593 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.602684 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 19:34:33.603932 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.603557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45724)
0926 19:34:33.603715 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.603806 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 19:34:33.618937 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 17663
Sep 26 19:34:33 dist-test-slave-phbh krb5kdc[17290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:34:33.636435 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.619737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37914 (local address 127.6.215.195:33633)
0926 19:34:33.619920 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:34:33.619925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:34:33.619945 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:34:33.620132 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:34:33.620136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:34:33.620210 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:34:33.620291 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:34:33.622114 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.622843 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.624646 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:34:33.625174 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:34:33.625278 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:34:33.630373 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:34:33.630386 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:34:33.630400 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:34:33.630442 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:34:33.633615 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:33.634302 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:33.634317 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:33.634319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:33.634446 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:34:33.634823 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:34:33.634829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:34:33.634833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:34:33.635213 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:34:33.635562 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0926 19:34:33.635872 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 19:34:33.636116 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 19:34:33.637166 17786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:33.685184 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.684612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45728)
0926 19:34:33.684891 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.684999 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260926 19:34:33.686417 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.685965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45744)
0926 19:34:33.686180 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.686259 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 19:34:33.757057 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 17663
I20260926 19:34:33.778774 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 17303
I20260926 19:34:33.779961 17365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:34:33.886340 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:34:33.885553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35595 (local address 127.0.0.1:45748)
0926 19:34:33.885989 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 19:34:33.886111 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 19:34:33.886519 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 17303
2026-09-26T19:34:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4552 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:34:33 dist-test-slave-phbh krb5kdc[17847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:34:33 dist-test-slave-phbh krb5kdc[17847](info): set up 2 sockets
Sep 26 19:34:33 dist-test-slave-phbh krb5kdc[17847](info): commencing operation
krb5kdc: starting...
W20260926 19:34:35.009752 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:34:35 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:34:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:34:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/rangeradmin_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/rangeradmin_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/rangerlookup_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/rangerlookup_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
I20260926 19:34:35.331297 7007 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taske_ElXL/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/postgres -l logfile start
2026-09-26 19:34:37.875 UTC [17877] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 19:34:37.875 UTC [17877] LOG: listening on IPv4 address "127.6.215.212", port 35721
2026-09-26 19:34:37.879 UTC [17877] LOG: listening on Unix socket "/tmp/.s.PGSQL.35721"
2026-09-26 19:34:37.901 UTC [17880] LOG: database system was shut down at 2026-09-26 19:34:36 UTC
2026-09-26 19:34:37.904 UTC [17877] LOG: database system is ready to accept connections
I20260926 19:34:38.885638 7007 mini_postgres.cc:96] Postgres bound to 35721
2026-09-26 19:34:38.908 UTC [17888] FATAL: database "slave" does not exist
127.6.215.212:35721 - accepting connections
I20260926 19:34:38.910002 7007 mini_ranger.cc:161] Starting Ranger...
I20260926 19:34:38.938966 7007 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 19:34:39.436602 7007 mini_ranger.cc:87] Created ranger Postgres database
I20260926 19:34:39.436710 7007 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-admin
2026-09-26 19:34:39,723 [I] DB FLAVOR :POSTGRES
2026-09-26 19:34:39,724 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 19:34:39,724 [I] Checking connection..
2026-09-26 19:34:39,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 19:34:40,005 [I] Checking connection passed.
2026-09-26 19:34:40,005 [I] --------- Verifying version history table ---------
2026-09-26 19:34:40,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 19:34:40,266 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 19:34:40,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 19:34:40,516 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 19:34:40,516 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 19:34:40,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 19:34:40.748 UTC [17952] WARNING: there is no transaction in progress
2026-09-26 19:34:40,755 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 19:34:40,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 19:34:41,019 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 19:34:41,020 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 19:34:41,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 19:34:41,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 19:34:41,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-phbh.c.gcp-upstream.internal','N') ;"
2026-09-26 19:34:41,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 19:34:41,985 [I] Table x_portal_user does not exist in database ranger
2026-09-26 19:34:41,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 19:34:42,223 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 19:34:42,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 19:34:42,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 19:34:42,722 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 19:34:42,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 19:34:43.672 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.689 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.705 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.721 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.737 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.753 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.807 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.814 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.828 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.838 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.849 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.865 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.875 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.881 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.889 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:43.895 UTC [18060] WARNING: there is no transaction in progress
2026-09-26 19:34:44,537 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 19:34:44,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 19:34:44,756 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 19:34:44,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 19:34:44,985 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 19:34:44,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 19:34:45,231 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 19:34:45,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 19:34:45,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 19:34:45,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-phbh.c.gcp-upstream.internal';"
2026-09-26 19:34:45,944 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 19:34:45,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 19:34:46,201 [I] DB_PATCHES have already been applied
I20260926 19:34:46.207376 7007 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-admin:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taske_ElXL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 19:34:46.207470 7007 mini_ranger.cc:193] Using host: 127.6.215.212
I20260926 19:34:46.222337 7007 mini_ranger.cc:227] Ranger admin URL: http://127.6.215.212:37987
Sep 26, 2026 7:34:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:34:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:34:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 7:34:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 7:34:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 7:34:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 7:34:47 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 19:34:47.375262 7007 mini_ranger.cc:160] Time spent starting Ranger: real 8.465s user 0.007s sys 0.036s
Sep 26, 2026 7:35:07 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 7:35:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 7:35:07 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 7:35:07 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 7:35:08 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 19:35:09.252980 7007 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/rangerkms_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/rangerkms_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.215.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.215.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 19:35:09 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451309, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 19:35:09.427942 7007 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 19:35:09.450852 7007 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 19:35:10.366973 7007 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 19:35:10.367132 7007 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms
I20260926 19:35:10.367274 7007 mini_ranger_kms.cc:227] Using postgres at 127.6.215.212:35721
2026-09-26 19:35:10,781 [I] DB FLAVOR :POSTGRES
2026-09-26 19:35:10,781 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 19:35:10,781 [I] Checking connection
2026-09-26 19:35:10,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 19:35:11,012 [I] connection success
2026-09-26 19:35:11,012 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 19:35:11,012 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 19:35:11,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 19:35:11,249 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 19:35:11,249 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 19:35:11,249 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 19:35:11,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:35721/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 19:35:11,510 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 19:35:11.960114 7007 mini_ranger_kms.cc:310] Created kms service
I20260926 19:35:12.122946 7007 mini_ranger_kms.cc:326] Created kudu user
I20260926 19:35:12.167748 7007 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 19:35:12 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451309, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:35:12.573958 7007 mini_ranger_kms.cc:384] Added ranger policy
I20260926 19:35:12.574323 7007 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms:/tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 19:35:12.574378 7007 mini_ranger_kms.cc:240] Using host: 127.6.215.212
I20260926 19:35:12.595811 7007 mini_ranger_kms.cc:276] Ranger KMS PID: 18240
I20260926 19:35:12.596009 7007 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.215.212:34463
Sep 26, 2026 7:35:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:35:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:35:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 7:35:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 7:35:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 7:35:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 19:35:14.899211 7007 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.471s user 0.102s sys 0.342s
I20260926 19:35:14.899462 7007 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 19:35:14.899675 7007 mini_ranger_kms.cc:401] 127.6.215.212:34463/kms/v1/keys
Sep 26 19:35:15 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790451315, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.215.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 19:35:15 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451315, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.215.212@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
Sep 26 19:35:15 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451315, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.215.212@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
Sep 26 19:35:18 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:35:18.194602 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:41813
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:41813
--ranger_config_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:34463/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:35:18.541487 18291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:35:18.541821 18291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:35:18.541972 18291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:35:18.549598 18291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:35:18.549754 18291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:35:18.549868 18291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:35:18.549961 18291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:35:18.550042 18291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:35:18.563310 18291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.215.212:34463/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:41813
--ranger_config_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:41813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:35:18.564626 18291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:35:18.565971 18291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:35:18.580276 18297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:35:18.580209 18296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:35:18.580250 18299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:35:18.581699 18291 server_base.cc:1034] running on GCE node
I20260926 19:35:18.582647 18291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:35:18.584494 18291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:35:18.585752 18291 hybrid_clock.cc:648] HybridClock initialized: now 1790451318585678 us; error 65 us; skew 500 ppm
I20260926 19:35:18.586154 18291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:35:18 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451318, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:35:18.594497 18291 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:35:18.596517 18291 webserver.cc:460] Webserver started at http://127.6.215.254:36291/ using document root <none> and password file <none>
I20260926 19:35:18.597291 18291 fs_manager.cc:359] Metadata directory not provided
I20260926 19:35:18.597389 18291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:35:18.597707 18291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:35:18 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451318, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:35:18.655493 18291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "bcea7a636a1449e5840535eb9d9a5a13"
format_stamp: "Formatted at 2026-09-26 19:35:18 on dist-test-slave-phbh"
server_key: "b710a6d483708de132cee32f57d6a69a"
server_key_iv: "4a4d67cc12fa5c38272030a5f4660e6e"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:18.656378 18291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "bcea7a636a1449e5840535eb9d9a5a13"
format_stamp: "Formatted at 2026-09-26 19:35:18 on dist-test-slave-phbh"
server_key: "b710a6d483708de132cee32f57d6a69a"
server_key_iv: "4a4d67cc12fa5c38272030a5f4660e6e"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:18.661821 18291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 19:35:18.681084 18306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:18.682866 18291 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 19:35:18.683055 18291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "bcea7a636a1449e5840535eb9d9a5a13"
format_stamp: "Formatted at 2026-09-26 19:35:18 on dist-test-slave-phbh"
server_key: "b710a6d483708de132cee32f57d6a69a"
server_key_iv: "4a4d67cc12fa5c38272030a5f4660e6e"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:18.683300 18291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:35:18.704986 18291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:35:18.709961 18291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:35:18.710271 18291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:35:18.731047 18357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:41813 every 8 connection(s)
I20260926 19:35:18.731056 18291 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:41813
I20260926 19:35:18.732929 18291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:35:18.737213 18358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:35:18.742733 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 18291
I20260926 19:35:18.742946 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:35:18.743777 7007 external_mini_cluster.cc:1428] Setting key 9d3a8cfea95aa7cb18e4c9057dfc8cb0
I20260926 19:35:18.752259 18358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13: Bootstrap starting.
I20260926 19:35:18.758423 18358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13: Neither blocks nor log segments found. Creating new log.
I20260926 19:35:18.760294 18358 log.cc:824] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13: Log is configured to *not* fsync() on all Append() calls
I20260926 19:35:18.763676 18358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13: No bootstrap required, opened a new log
I20260926 19:35:18.775120 18358 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea7a636a1449e5840535eb9d9a5a13" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41813 } }
Sep 26 19:35:18 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:35:18.775529 18358 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:35:18.775625 18358 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcea7a636a1449e5840535eb9d9a5a13, State: Initialized, Role: FOLLOWER
I20260926 19:35:18.776301 18358 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea7a636a1449e5840535eb9d9a5a13" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41813 } }
I20260926 19:35:18.776558 18358 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:35:18.776751 18358 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:35:18.777097 18358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:35:18.780004 18358 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea7a636a1449e5840535eb9d9a5a13" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41813 } }
I20260926 19:35:18.780581 18358 leader_election.cc:302] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcea7a636a1449e5840535eb9d9a5a13; no voters:
I20260926 19:35:18.781396 18358 leader_election.cc:288] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:35:18.781481 18363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:35:18.787817 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:18.746119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.6.215.254:41813)
0926 19:35:18.760821 (+ 14702us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:18.760837 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:18.760883 (+ 46us) server_negotiation.cc:407] Connection header received
0926 19:35:18.763347 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:18.763378 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:18.763872 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:18.764318 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:18.766137 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:18.768130 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:18.770260 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:18.771392 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:18.772368 (+ 976us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:18.776800 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:18.776813 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:18.776833 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:18.776877 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:18.780744 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:18.781802 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:18.781815 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:18.781828 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:18.782086 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:18.782752 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:18.782761 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:18.782768 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:18.783762 (+ 994us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:18.784141 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:18.785175 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0926 19:35:18.785789 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14433,"thread_start_us":14201,"threads_started":1}
I20260926 19:35:18.788589 18363 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [term 1 LEADER]: Becoming Leader. State: Replica: bcea7a636a1449e5840535eb9d9a5a13, State: Running, Role: LEADER
I20260926 19:35:18.789281 18363 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea7a636a1449e5840535eb9d9a5a13" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41813 } }
I20260926 19:35:18.790331 18358 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:35:18.795441 18365 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcea7a636a1449e5840535eb9d9a5a13. Latest consensus state: current_term: 1 leader_uuid: "bcea7a636a1449e5840535eb9d9a5a13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea7a636a1449e5840535eb9d9a5a13" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41813 } } }
I20260926 19:35:18.796376 18365 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [sys.catalog]: This master's current role is: LEADER
I20260926 19:35:18.796096 18364 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcea7a636a1449e5840535eb9d9a5a13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea7a636a1449e5840535eb9d9a5a13" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41813 } } }
I20260926 19:35:18.796758 18364 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13 [sys.catalog]: This master's current role is: LEADER
I20260926 19:35:18.801128 18369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:35:18.808146 18369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:35:18.809031 18358 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 19:35:18.831867 18369 catalog_manager.cc:1347] Generated new cluster ID: 956c89f78f554b11b1231f1682086af0
I20260926 19:35:18.832259 18369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:35:18.856720 18369 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:35:18.859699 18369 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:35:18.872507 18369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13: Generated new TSK 0
I20260926 19:35:18.875458 18369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:35:19.777278 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:34463/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41813
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:35:20 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790451320, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 19:35:20.196815 18403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:35:20.197196 18403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:35:20.197356 18403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:35:20.205976 18403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:35:20.206202 18403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:35:20.206427 18403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:35:20.222750 18403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.215.212:34463/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:35:20.224646 18403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:35:20.226384 18403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:35:20.244920 18413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:35:20.245510 18414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:35:20.254582 18403 server_base.cc:1034] running on GCE node
W20260926 19:35:20.255023 18416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:35:20.255909 18403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:35:20.257728 18403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:35:20.259048 18403 hybrid_clock.cc:648] HybridClock initialized: now 1790451320258981 us; error 54 us; skew 500 ppm
I20260926 19:35:20.259461 18403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:35:20 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451320, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:35:20.263815 18403 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:35:20.265868 18403 webserver.cc:460] Webserver started at http://127.6.215.193:32943/ using document root <none> and password file <none>
I20260926 19:35:20.266808 18403 fs_manager.cc:359] Metadata directory not provided
I20260926 19:35:20.266997 18403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:35:20.267364 18403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:35:20 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451320, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:35:20.289403 18403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "b01c92e64c8a49068bbd8a728c4cdafa"
format_stamp: "Formatted at 2026-09-26 19:35:20 on dist-test-slave-phbh"
server_key: "a2f019874d2c181eec587773df56fd3b"
server_key_iv: "b26ae3f6cf52aa3d5da4608c501902ff"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:20.290630 18403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "b01c92e64c8a49068bbd8a728c4cdafa"
format_stamp: "Formatted at 2026-09-26 19:35:20 on dist-test-slave-phbh"
server_key: "a2f019874d2c181eec587773df56fd3b"
server_key_iv: "b26ae3f6cf52aa3d5da4608c501902ff"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:20.304570 18403 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.004s
I20260926 19:35:20.328868 18423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:20.331166 18403 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260926 19:35:20.331401 18403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "b01c92e64c8a49068bbd8a728c4cdafa"
format_stamp: "Formatted at 2026-09-26 19:35:20 on dist-test-slave-phbh"
server_key: "a2f019874d2c181eec587773df56fd3b"
server_key_iv: "b26ae3f6cf52aa3d5da4608c501902ff"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:20.331698 18403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:35:20.356210 18403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:35:20.361621 18403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:35:20.362067 18403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:35:20.368875 18403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:35:20.376210 18403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:35:20.376328 18403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:20.376467 18403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:35:20.376559 18403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:20.453554 18403 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:46175
I20260926 19:35:20.453594 18535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:46175 every 8 connection(s)
I20260926 19:35:20.455377 18403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:35:20.455900 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 18403
I20260926 19:35:20.456122 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:35:20.457118 7007 external_mini_cluster.cc:1428] Setting key 88da33ad67063234c6725d59f57cd711
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Sep 26 19:35:20 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451320, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:35:20.537631 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:20.480147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:44933 (local address 127.6.215.254:41813)
0926 19:35:20.506141 (+ 25994us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:20.506148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:20.506177 (+ 29us) server_negotiation.cc:407] Connection header received
0926 19:35:20.511775 (+ 5598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:20.511782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:20.511900 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:20.512000 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:20.516970 (+ 4970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:20.518152 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:20.520655 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:20.521791 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:20.521928 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:20.528962 (+ 7034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:20.528970 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:20.528975 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:20.529023 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:20.532574 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:20.533656 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:20.533662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:20.533666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:20.533810 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:20.534325 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:20.534331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:20.534334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:20.534691 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:20.534823 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:20.536980 (+ 2157us) server_negotiation.cc:299] Negotiation successful
0926 19:35:20.537217 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25841,"thread_start_us":26011,"threads_started":1}
I20260926 19:35:20.538744 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:20.481184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41813 (local address 127.6.215.193:44933)
0926 19:35:20.501665 (+ 20481us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:20.501740 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:20.504561 (+ 2821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:20.512685 (+ 8124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:20.512700 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:20.514059 (+ 1359us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:35:20.514991 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:20.515019 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:20.518395 (+ 3376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:20.518400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:20.520461 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:20.520491 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:20.522075 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:20.522082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:20.522438 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:20.523557 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:35:20.523588 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:35:20.526339 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:35:20.532817 (+ 6478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:35:20.532828 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:35:20.532849 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:35:20.533485 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:35:20.533937 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:35:20.533942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:35:20.533946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:35:20.534195 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:35:20.534902 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:35:20.534910 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:35:20.535349 (+ 439us) client_negotiation.cc:769] Sending connection context
0926 19:35:20.535662 (+ 313us) client_negotiation.cc:240] Negotiation successful
0926 19:35:20.536210 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20048,"thread_start_us":23479,"threads_started":1}
I20260926 19:35:20.541226 18536 heartbeater.cc:347] Connected to a master server at 127.6.215.254:41813
I20260926 19:35:20.541599 18536 heartbeater.cc:464] Registering TS with master...
I20260926 19:35:20.542500 18536 heartbeater.cc:511] Master 127.6.215.254:41813 requested a full tablet report, sending...
I20260926 19:35:20.545087 18323 ts_manager.cc:194] Registered new tserver with Master: b01c92e64c8a49068bbd8a728c4cdafa (127.6.215.193:46175)
I20260926 19:35:20.547108 18323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:44933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
W20260926 19:35:20.573638 7007 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.6.215.194: real 0.113s user 0.003s sys 0.014s
I20260926 19:35:20.574605 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:34463/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41813
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:35:20 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451320, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
Sep 26 19:35:20 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451320, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
W20260926 19:35:21.044066 18542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:35:21.044629 18542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:35:21.044871 18542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:35:21.055420 18542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:35:21.055590 18542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:35:21.055799 18542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:35:21.069082 18542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.215.212:34463/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:35:21.070746 18542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:35:21.072608 18542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:35:21.115475 18550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:35:21.124383 18549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:35:21.129241 18542 server_base.cc:1034] running on GCE node
W20260926 19:35:21.133100 18552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:35:21.134140 18542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:35:21.135771 18542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:35:21.137033 18542 hybrid_clock.cc:648] HybridClock initialized: now 1790451321136979 us; error 55 us; skew 500 ppm
I20260926 19:35:21.137473 18542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:35:21 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:35:21.141449 18542 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:35:21.146651 18542 webserver.cc:460] Webserver started at http://127.6.215.194:37451/ using document root <none> and password file <none>
I20260926 19:35:21.147409 18542 fs_manager.cc:359] Metadata directory not provided
I20260926 19:35:21.147511 18542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:35:21.147848 18542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:35:21 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:35:21.165688 18542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "ea2db21471f145649b2de067f4df12a4"
format_stamp: "Formatted at 2026-09-26 19:35:21 on dist-test-slave-phbh"
server_key: "26bc4175325a4bbf2e368a1c789b0c74"
server_key_iv: "8e1df1edfe49945465acd390cd162d72"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:21.166962 18542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "ea2db21471f145649b2de067f4df12a4"
format_stamp: "Formatted at 2026-09-26 19:35:21 on dist-test-slave-phbh"
server_key: "26bc4175325a4bbf2e368a1c789b0c74"
server_key_iv: "8e1df1edfe49945465acd390cd162d72"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:21.177151 18542 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.000s
I20260926 19:35:21.198199 18566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:21.199895 18542 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260926 19:35:21.200110 18542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "ea2db21471f145649b2de067f4df12a4"
format_stamp: "Formatted at 2026-09-26 19:35:21 on dist-test-slave-phbh"
server_key: "26bc4175325a4bbf2e368a1c789b0c74"
server_key_iv: "8e1df1edfe49945465acd390cd162d72"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:21.200356 18542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:35:21.239526 18542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:35:21.243651 18542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:35:21.243958 18542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:35:21.244972 18542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:35:21.246337 18542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:35:21.246440 18542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:21.246580 18542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:35:21.246649 18542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:21.291088 18542 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:36829
I20260926 19:35:21.291106 18678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:36829 every 8 connection(s)
I20260926 19:35:21.292519 18542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:35:21.297544 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 18542
I20260926 19:35:21.297734 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:35:21.298162 7007 external_mini_cluster.cc:1428] Setting key 0c966b5f18706195041ca03652b1265e
Sep 26 19:35:21 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:35:21.316248 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.295294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:44847 (local address 127.6.215.254:41813)
0926 19:35:21.301648 (+ 6354us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:21.301655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:21.301676 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:35:21.301731 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:21.301736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:21.301804 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:21.301929 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:21.303538 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.304356 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.306112 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.306516 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.306662 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.310200 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:21.310210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:21.310215 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:21.310258 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:21.313225 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:21.313954 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:21.313965 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:21.313967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:21.314081 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:21.314513 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:21.314518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:21.314521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:21.314799 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:21.314948 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:21.315642 (+ 694us) server_negotiation.cc:299] Negotiation successful
0926 19:35:21.315862 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6205,"thread_start_us":6074,"threads_started":1}
I20260926 19:35:21.317313 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.296206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41813 (local address 127.6.215.194:44847)
0926 19:35:21.297174 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:21.297224 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:21.299428 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:21.302212 (+ 2784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:21.302221 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:21.302672 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:35:21.303348 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.303367 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.304507 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.304512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.305961 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.305972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.306745 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.306752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.306978 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.307664 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:35:21.307687 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:35:21.309972 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:35:21.313368 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:35:21.313377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:35:21.313395 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:35:21.313804 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:35:21.314195 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:35:21.314202 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:35:21.314205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:35:21.314397 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:35:21.314922 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:35:21.314931 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:35:21.315272 (+ 341us) client_negotiation.cc:769] Sending connection context
0926 19:35:21.315534 (+ 262us) client_negotiation.cc:240] Negotiation successful
0926 19:35:21.316071 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":395,"threads_started":1}
I20260926 19:35:21.318830 18679 heartbeater.cc:347] Connected to a master server at 127.6.215.254:41813
I20260926 19:35:21.319144 18679 heartbeater.cc:464] Registering TS with master...
I20260926 19:35:21.319852 18679 heartbeater.cc:511] Master 127.6.215.254:41813 requested a full tablet report, sending...
I20260926 19:35:21.321316 18323 ts_manager.cc:194] Registered new tserver with Master: ea2db21471f145649b2de067f4df12a4 (127.6.215.194:36829)
I20260926 19:35:21.322117 18323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:44847
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:35:21.362709 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:34463/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41813
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:45811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:35:21.551206 18536 heartbeater.cc:503] Master 127.6.215.254:41813 was elected leader, sending a full tablet report...
W20260926 19:35:21.719996 18685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:35:21.720336 18685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:35:21.720508 18685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:35:21.727710 18685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:35:21.727867 18685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:35:21.728078 18685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:35:21.738667 18685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:45811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.215.212:34463/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:41813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:35:21.739913 18685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:35:21.741215 18685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:35:21.754437 18690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:35:21.754405 18693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:35:21.754333 18691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:35:21.755717 18685 server_base.cc:1034] running on GCE node
I20260926 19:35:21.756366 18685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:35:21.757395 18685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:35:21.758599 18685 hybrid_clock.cc:648] HybridClock initialized: now 1790451321758559 us; error 41 us; skew 500 ppm
I20260926 19:35:21.758924 18685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:35:21 dist-test-slave-phbh krb5kdc[17847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:35:21.762173 18685 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:35:21.763645 18685 webserver.cc:460] Webserver started at http://127.6.215.195:37453/ using document root <none> and password file <none>
I20260926 19:35:21.764351 18685 fs_manager.cc:359] Metadata directory not provided
I20260926 19:35:21.764480 18685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:35:21.764753 18685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:35:21 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:35:21.778105 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "efef9386b20f4b8b92798acdd97da271"
format_stamp: "Formatted at 2026-09-26 19:35:21 on dist-test-slave-phbh"
server_key: "81d5c8de7d6bc80db482502f006afd0c"
server_key_iv: "c47c6732cec16a53b6acdd86e7028dc4"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:21.779042 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "efef9386b20f4b8b92798acdd97da271"
format_stamp: "Formatted at 2026-09-26 19:35:21 on dist-test-slave-phbh"
server_key: "81d5c8de7d6bc80db482502f006afd0c"
server_key_iv: "c47c6732cec16a53b6acdd86e7028dc4"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:21.784551 18685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 19:35:21.801385 18700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:21.803212 18685 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 19:35:21.803398 18685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "efef9386b20f4b8b92798acdd97da271"
format_stamp: "Formatted at 2026-09-26 19:35:21 on dist-test-slave-phbh"
server_key: "81d5c8de7d6bc80db482502f006afd0c"
server_key_iv: "c47c6732cec16a53b6acdd86e7028dc4"
server_key_version: "kuduclusterkey@0"
I20260926 19:35:21.803665 18685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:35:21.825903 18685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:35:21.830915 18685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:35:21.831317 18685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:35:21.832517 18685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:35:21.834813 18685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:35:21.834908 18685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:21.835018 18685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:35:21.835078 18685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:35:21.871694 18685 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:45379
I20260926 19:35:21.871712 18812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:45379 every 8 connection(s)
I20260926 19:35:21.873330 18685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:35:21.882618 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 18685
I20260926 19:35:21.882814 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:35:21.883287 7007 external_mini_cluster.cc:1428] Setting key abffe2f45741e2279ea87a052a40d726
Sep 26 19:35:21 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:35:21.896687 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.876136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:41259 (local address 127.6.215.254:41813)
0926 19:35:21.876627 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:21.876631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:21.877733 (+ 1102us) server_negotiation.cc:407] Connection header received
0926 19:35:21.879620 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:21.879625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:21.879724 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:21.879854 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:21.881802 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.882793 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.885229 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.886347 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.886484 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.890654 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:21.890661 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:21.890665 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:21.890696 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:21.893618 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:21.894350 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:21.894356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:21.894360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:21.894482 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:21.894960 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:21.894965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:21.894969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:21.895247 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:21.895403 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:21.896110 (+ 707us) server_negotiation.cc:299] Negotiation successful
0926 19:35:21.896322 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":277,"threads_started":1}
I20260926 19:35:21.898010 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.876736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41813 (local address 127.6.215.195:41259)
0926 19:35:21.877556 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:21.877605 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:21.879308 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:21.880159 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:21.880171 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:21.880755 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:35:21.881585 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.881610 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.882990 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.882995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.885069 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.885085 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.886593 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.886601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.886908 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.887739 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:35:21.887770 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:35:21.890361 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:35:21.893761 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:35:21.893771 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:35:21.893789 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:35:21.894220 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:35:21.894603 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:35:21.894608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:35:21.894611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:35:21.894835 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:35:21.895378 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:35:21.895386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:35:21.895735 (+ 349us) client_negotiation.cc:769] Sending connection context
0926 19:35:21.896019 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 19:35:21.896569 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":363,"threads_started":1}
I20260926 19:35:21.899731 18813 heartbeater.cc:347] Connected to a master server at 127.6.215.254:41813
I20260926 19:35:21.900070 18813 heartbeater.cc:464] Registering TS with master...
I20260926 19:35:21.900815 18813 heartbeater.cc:511] Master 127.6.215.254:41813 requested a full tablet report, sending...
I20260926 19:35:21.902366 18323 ts_manager.cc:194] Registered new tserver with Master: efef9386b20f4b8b92798acdd97da271 (127.6.215.195:45379)
I20260926 19:35:21.903136 18323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:41259
I20260926 19:35:21.912111 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:35:21.928896 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.916474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.6.215.254:41813)
0926 19:35:21.916671 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:21.916679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:21.917167 (+ 488us) server_negotiation.cc:407] Connection header received
0926 19:35:21.917331 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:21.917336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:21.917414 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:21.917512 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:21.919261 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.919972 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.921303 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.921721 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.921820 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.924034 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:21.924039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:21.924042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:21.924072 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:21.926568 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:21.927177 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:21.927184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:21.927188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:21.927310 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:21.927609 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:21.927612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:21.927614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:21.927850 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:21.928037 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:21.928316 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 19:35:21.928495 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 19:35:21.933038 18323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:35:21.940310 18323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:35:21.990124 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.953710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.6.215.194:36829)
0926 19:35:21.976535 (+ 22825us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:21.976550 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:21.978274 (+ 1724us) server_negotiation.cc:407] Connection header received
0926 19:35:21.978401 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:21.978413 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:21.978769 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:21.978987 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:35:21.980210 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.982355 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.987372 (+ 5017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.989003 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.989139 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.989334 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:21.989635 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 19:35:21.989800 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22546,"thread_start_us":22856,"threads_started":1}
I20260926 19:35:21.990144 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.952613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:36829 (local address 127.0.0.1:45856)
0926 19:35:21.977673 (+ 25060us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:21.977743 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:21.977970 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:21.979319 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:21.979330 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:21.979373 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:35:21.979956 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.979970 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.983357 (+ 3387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.983366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.987146 (+ 3780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.987171 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.989206 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.989213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.989409 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.989464 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 19:35:21.989596 (+ 132us) client_negotiation.cc:240] Negotiation successful
0926 19:35:21.989810 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24697,"thread_start_us":34615,"threads_started":1}
I20260926 19:35:21.995100 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.952704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:45379 (local address 127.0.0.1:48334)
0926 19:35:21.987157 (+ 34453us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:21.987180 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:21.987330 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:21.988049 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:21.988053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:21.988077 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:35:21.988488 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.988505 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.990530 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.990536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.992462 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.992476 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.994324 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.994330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.994489 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.994520 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:35:21.994625 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 19:35:21.994752 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34311,"thread_start_us":34170,"threads_started":1}
I20260926 19:35:21.995195 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.953064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.6.215.195:45379)
0926 19:35:21.986056 (+ 32992us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:21.986066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:21.987490 (+ 1424us) server_negotiation.cc:407] Connection header received
0926 19:35:21.987602 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:21.987611 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:21.987864 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:21.988065 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:35:21.988730 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.990370 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.992615 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.994113 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.994243 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.994468 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:21.994672 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 19:35:21.994838 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32810,"thread_start_us":32661,"threads_started":1}
I20260926 19:35:21.994762 18614 tablet_service.cc:1467] Processing CreateTablet for tablet f936bb575c6347f6bcb315e510dfc204 (DEFAULT_TABLE table=test-table [id=1901659f47854dc68519b18e6c5bce3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:35:21.995796 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.953097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:46175 (local address 127.0.0.1:60900)
0926 19:35:21.987500 (+ 34403us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:21.987523 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:21.987626 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:21.988420 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:21.988428 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:21.988458 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:35:21.988882 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.988891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.990896 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.990901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.992745 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:21.992758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.994800 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.994806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:21.994975 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.995013 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 19:35:21.995110 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 19:35:21.995409 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34281,"mutex_wait_us":1,"thread_start_us":29414,"threads_started":1}
I20260926 19:35:21.997017 18614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f936bb575c6347f6bcb315e510dfc204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:35:21.998332 18748 tablet_service.cc:1467] Processing CreateTablet for tablet f936bb575c6347f6bcb315e510dfc204 (DEFAULT_TABLE table=test-table [id=1901659f47854dc68519b18e6c5bce3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:35:21.999351 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:21.953831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.6.215.193:46175)
0926 19:35:21.986470 (+ 32639us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:21.986481 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:21.987682 (+ 1201us) server_negotiation.cc:407] Connection header received
0926 19:35:21.987795 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:21.987802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:21.988037 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:21.988227 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:35:21.989051 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.990751 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.992964 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:21.994296 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:21.994414 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:21.994652 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:21.996010 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0926 19:35:21.999007 (+ 2997us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32423,"thread_start_us":32641,"threads_started":1}
I20260926 19:35:21.999519 18748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f936bb575c6347f6bcb315e510dfc204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:35:22.002853 18471 tablet_service.cc:1467] Processing CreateTablet for tablet f936bb575c6347f6bcb315e510dfc204 (DEFAULT_TABLE table=test-table [id=1901659f47854dc68519b18e6c5bce3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:35:22.004761 18471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f936bb575c6347f6bcb315e510dfc204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:35:22.009294 18832 tablet_bootstrap.cc:492] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Bootstrap starting.
I20260926 19:35:22.012387 18832 tablet_bootstrap.cc:654] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Neither blocks nor log segments found. Creating new log.
I20260926 19:35:22.013657 18832 log.cc:824] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Log is configured to *not* fsync() on all Append() calls
I20260926 19:35:22.017695 18833 tablet_bootstrap.cc:492] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Bootstrap starting.
I20260926 19:35:22.020565 18832 tablet_bootstrap.cc:492] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: No bootstrap required, opened a new log
I20260926 19:35:22.020871 18832 ts_tablet_manager.cc:1397] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260926 19:35:22.022222 18833 tablet_bootstrap.cc:654] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Neither blocks nor log segments found. Creating new log.
I20260926 19:35:22.023458 18833 log.cc:824] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Log is configured to *not* fsync() on all Append() calls
I20260926 19:35:22.024581 18835 tablet_bootstrap.cc:492] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Bootstrap starting.
I20260926 19:35:22.027518 18835 tablet_bootstrap.cc:654] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Neither blocks nor log segments found. Creating new log.
I20260926 19:35:22.028692 18835 log.cc:824] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Log is configured to *not* fsync() on all Append() calls
I20260926 19:35:22.030097 18833 tablet_bootstrap.cc:492] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: No bootstrap required, opened a new log
I20260926 19:35:22.030395 18833 ts_tablet_manager.cc:1397] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260926 19:35:22.031211 18835 tablet_bootstrap.cc:492] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: No bootstrap required, opened a new log
I20260926 19:35:22.031452 18835 ts_tablet_manager.cc:1397] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260926 19:35:22.032792 18832 raft_consensus.cc:348] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.033224 18832 raft_consensus.cc:374] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:35:22.033391 18832 raft_consensus.cc:729] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2db21471f145649b2de067f4df12a4, State: Initialized, Role: FOLLOWER
I20260926 19:35:22.034014 18832 consensus_queue.cc:260] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.035121 18679 heartbeater.cc:503] Master 127.6.215.254:41813 was elected leader, sending a full tablet report...
I20260926 19:35:22.035723 18832 ts_tablet_manager.cc:1428] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260926 19:35:22.040820 18833 raft_consensus.cc:348] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.041292 18833 raft_consensus.cc:374] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:35:22.041401 18833 raft_consensus.cc:729] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efef9386b20f4b8b92798acdd97da271, State: Initialized, Role: FOLLOWER
I20260926 19:35:22.041569 18835 raft_consensus.cc:348] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.041939 18835 raft_consensus.cc:374] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:35:22.042057 18835 raft_consensus.cc:729] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b01c92e64c8a49068bbd8a728c4cdafa, State: Initialized, Role: FOLLOWER
I20260926 19:35:22.042032 18833 consensus_queue.cc:260] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.042650 18835 consensus_queue.cc:260] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.043202 18813 heartbeater.cc:503] Master 127.6.215.254:41813 was elected leader, sending a full tablet report...
I20260926 19:35:22.043829 18833 ts_tablet_manager.cc:1428] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
W20260926 19:35:22.044608 18680 tablet.cc:2369] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:35:22.044697 18835 ts_tablet_manager.cc:1428] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
W20260926 19:35:22.125154 18814 tablet.cc:2369] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:35:22.127571 18839 raft_consensus.cc:482] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:35:22.127868 18839 raft_consensus.cc:504] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.129515 18839 leader_election.cc:288] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea2db21471f145649b2de067f4df12a4 (127.6.215.194:36829), b01c92e64c8a49068bbd8a728c4cdafa (127.6.215.193:46175)
I20260926 19:35:22.136149 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.130042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:60361 (local address 127.6.215.193:46175)
0926 19:35:22.130239 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.130246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.130275 (+ 29us) server_negotiation.cc:407] Connection header received
0926 19:35:22.130619 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.130627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.130730 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.130823 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:35:22.131407 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.132375 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.134258 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.135240 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.135361 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.135412 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.135827 (+ 415us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.135927 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:35:22.136312 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.129861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:46175 (local address 127.6.215.195:60361)
0926 19:35:22.130165 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:22.130187 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:22.130358 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:22.130982 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:22.130986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:22.131009 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:35:22.131247 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.131259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.132547 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.132553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.134102 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.134113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.135418 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.135428 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.135649 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.135689 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 19:35:22.135810 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 19:35:22.135967 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":3}
I20260926 19:35:22.137061 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.129974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:36829 (local address 127.6.215.195:50329)
0926 19:35:22.131095 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:22.131116 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:22.131228 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:22.131547 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:22.131552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:22.131576 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:35:22.131899 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.131910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.133147 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.133150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.134654 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.134668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.136239 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.136245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.136438 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.136469 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:35:22.136562 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 19:35:22.136692 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"thread_start_us":819,"threads_started":1}
I20260926 19:35:22.137068 18840 raft_consensus.cc:482] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:35:22.137671 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.130155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:50329 (local address 127.6.215.194:36829)
0926 19:35:22.130365 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.130371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.131213 (+ 842us) server_negotiation.cc:407] Connection header received
0926 19:35:22.131352 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.131357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.131432 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.131501 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:35:22.132071 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.133000 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.134831 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.136014 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.136184 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.136255 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.136631 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.136832 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:35:22.137369 18840 raft_consensus.cc:504] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.137849 18491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f936bb575c6347f6bcb315e510dfc204" candidate_uuid: "efef9386b20f4b8b92798acdd97da271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" is_pre_election: true
I20260926 19:35:22.138383 18491 raft_consensus.cc:2454] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efef9386b20f4b8b92798acdd97da271 in term 0.
I20260926 19:35:22.138633 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f936bb575c6347f6bcb315e510dfc204" candidate_uuid: "efef9386b20f4b8b92798acdd97da271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2db21471f145649b2de067f4df12a4" is_pre_election: true
I20260926 19:35:22.139086 18702 leader_election.cc:302] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b01c92e64c8a49068bbd8a728c4cdafa, efef9386b20f4b8b92798acdd97da271; no voters:
I20260926 19:35:22.139168 18634 raft_consensus.cc:2454] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efef9386b20f4b8b92798acdd97da271 in term 0.
I20260926 19:35:22.139413 18839 raft_consensus.cc:2781] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:35:22.139552 18839 raft_consensus.cc:482] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:35:22.139650 18839 raft_consensus.cc:3037] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:35:22.139639 18840 leader_election.cc:288] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efef9386b20f4b8b92798acdd97da271 (127.6.215.195:45379), ea2db21471f145649b2de067f4df12a4 (127.6.215.194:36829)
I20260926 19:35:22.143615 18839 raft_consensus.cc:504] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.144330 18839 leader_election.cc:288] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [CANDIDATE]: Term 1 election: Requested vote from peers ea2db21471f145649b2de067f4df12a4 (127.6.215.194:36829), b01c92e64c8a49068bbd8a728c4cdafa (127.6.215.193:46175)
I20260926 19:35:22.144876 18491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f936bb575c6347f6bcb315e510dfc204" candidate_uuid: "efef9386b20f4b8b92798acdd97da271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b01c92e64c8a49068bbd8a728c4cdafa"
I20260926 19:35:22.145001 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f936bb575c6347f6bcb315e510dfc204" candidate_uuid: "efef9386b20f4b8b92798acdd97da271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2db21471f145649b2de067f4df12a4"
I20260926 19:35:22.145109 18491 raft_consensus.cc:3037] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:35:22.145242 18634 raft_consensus.cc:3037] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:35:22.148607 18634 raft_consensus.cc:2454] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efef9386b20f4b8b92798acdd97da271 in term 1.
I20260926 19:35:22.149212 18703 leader_election.cc:302] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea2db21471f145649b2de067f4df12a4, efef9386b20f4b8b92798acdd97da271; no voters:
I20260926 19:35:22.149731 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.140606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:45711 (local address 127.6.215.195:45379)
0926 19:35:22.140822 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.140828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.141427 (+ 599us) server_negotiation.cc:407] Connection header received
0926 19:35:22.141723 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.141730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.141806 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.141868 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:35:22.142648 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.143732 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.146551 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.148193 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.148324 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.148395 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.149320 (+ 925us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.149459 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:35:22.149564 18839 raft_consensus.cc:2781] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:35:22.149997 18491 raft_consensus.cc:2454] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efef9386b20f4b8b92798acdd97da271 in term 1.
I20260926 19:35:22.150221 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.139907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:45379 (local address 127.6.215.193:45711)
0926 19:35:22.141300 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:22.141320 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:22.141484 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:22.142089 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:22.142092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:22.142115 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:35:22.142498 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.142510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.143891 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.143896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.146358 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.146384 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.148933 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.148944 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.149100 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.149162 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 19:35:22.149605 (+ 443us) client_negotiation.cc:240] Negotiation successful
0926 19:35:22.149743 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"mutex_wait_us":8,"thread_start_us":1200,"threads_started":1}
I20260926 19:35:22.150358 18839 raft_consensus.cc:686] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 1 LEADER]: Becoming Leader. State: Replica: efef9386b20f4b8b92798acdd97da271, State: Running, Role: LEADER
I20260926 19:35:22.150952 18839 consensus_queue.cc:237] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } }
I20260926 19:35:22.151566 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.140682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:46313 (local address 127.6.215.194:36829)
0926 19:35:22.140860 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.140866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.142033 (+ 1167us) server_negotiation.cc:407] Connection header received
0926 19:35:22.142205 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.142212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.142290 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.142384 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:35:22.143040 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.144071 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.147033 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.148449 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.148613 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.148704 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.151250 (+ 2546us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.151386 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 19:35:22.151692 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.141102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:36829 (local address 127.6.215.193:46313)
0926 19:35:22.141916 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0926 19:35:22.141935 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:35:22.142073 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:35:22.142430 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:35:22.142446 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:35:22.142478 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:35:22.142865 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.142877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.144263 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.144269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.146831 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:35:22.146848 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.148867 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.148873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:35:22.149060 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.151102 (+ 2042us) client_negotiation.cc:769] Sending connection context
0926 19:35:22.151191 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 19:35:22.151325 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":2041,"thread_start_us":391,"threads_started":1}
I20260926 19:35:22.151645 18768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f936bb575c6347f6bcb315e510dfc204" candidate_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efef9386b20f4b8b92798acdd97da271" is_pre_election: true
I20260926 19:35:22.152271 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f936bb575c6347f6bcb315e510dfc204" candidate_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2db21471f145649b2de067f4df12a4" is_pre_election: true
I20260926 19:35:22.152547 18634 raft_consensus.cc:2379] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b01c92e64c8a49068bbd8a728c4cdafa in current term 1: Already voted for candidate efef9386b20f4b8b92798acdd97da271 in this term.
I20260926 19:35:22.153208 18426 leader_election.cc:302] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b01c92e64c8a49068bbd8a728c4cdafa; no voters: ea2db21471f145649b2de067f4df12a4, efef9386b20f4b8b92798acdd97da271
I20260926 19:35:22.153560 18840 raft_consensus.cc:2726] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 19:35:22.156888 18323 catalog_manager.cc:5495] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 reported cstate change: term changed from 0 to 1, leader changed from <none> to efef9386b20f4b8b92798acdd97da271 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "efef9386b20f4b8b92798acdd97da271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efef9386b20f4b8b92798acdd97da271" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 45379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 } health_report { overall_health: UNKNOWN } } }
I20260926 19:35:22.207543 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.201655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.6.215.195:45379)
0926 19:35:22.201829 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.201833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.201853 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:35:22.202155 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.202159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.202236 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.202361 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:35:22.202669 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.203850 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.205431 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.205859 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.205954 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.206429 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:35:22.206961 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:35:22.207072 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.207150 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.207213 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:35:22.215785 18634 raft_consensus.cc:1261] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 1 FOLLOWER]: Refusing update from remote peer efef9386b20f4b8b92798acdd97da271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:35:22.215824 18491 raft_consensus.cc:1261] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 1 FOLLOWER]: Refusing update from remote peer efef9386b20f4b8b92798acdd97da271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:35:22.216804 18844 consensus_queue.cc:1035] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2db21471f145649b2de067f4df12a4" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 36829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:35:22.217212 18839 consensus_queue.cc:1035] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b01c92e64c8a49068bbd8a728c4cdafa" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 46175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:35:22.230304 18848 mvcc.cc:205] Tried to move back new op lower bound from 7333688615785218048 to 7333688615539822592. Current Snapshot: MvccSnapshot[applied={T|T < 7333688615785218048}]
I20260926 19:35:22.243008 18322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:35:22.243343 18322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:35:22.247071 18322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcea7a636a1449e5840535eb9d9a5a13: Sending DeleteTablet for 3 replicas of tablet f936bb575c6347f6bcb315e510dfc204
I20260926 19:35:22.248168 18748 tablet_service.cc:1514] Processing DeleteTablet for tablet f936bb575c6347f6bcb315e510dfc204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:35:22 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:48334
I20260926 19:35:22.248324 18614 tablet_service.cc:1514] Processing DeleteTablet for tablet f936bb575c6347f6bcb315e510dfc204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:35:22 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45856
I20260926 19:35:22.248319 18471 tablet_service.cc:1514] Processing DeleteTablet for tablet f936bb575c6347f6bcb315e510dfc204 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:35:22 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:60900
I20260926 19:35:22.252449 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 18403
I20260926 19:35:22.259225 18857 tablet_replica.cc:330] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: stopping tablet replica
I20260926 19:35:22.259812 18857 raft_consensus.cc:2229] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:35:22.260442 18857 raft_consensus.cc:2258] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:35:22.262804 18856 tablet_replica.cc:330] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: stopping tablet replica
I20260926 19:35:22.263353 18856 raft_consensus.cc:2229] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:35:22.263743 18857 ts_tablet_manager.cc:1905] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:35:22.264091 18856 raft_consensus.cc:2258] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:35:22.264947 18858 tablet_replica.cc:330] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: stopping tablet replica
I20260926 19:35:22.265756 18858 raft_consensus.cc:2229] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:35:22.266271 18858 raft_consensus.cc:2258] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:35:22.266515 18856 ts_tablet_manager.cc:1905] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:35:22.268250 18858 ts_tablet_manager.cc:1905] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:35:22.271740 18857 ts_tablet_manager.cc:1918] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:35:22.271963 18857 log.cc:1196] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/f936bb575c6347f6bcb315e510dfc204
I20260926 19:35:22.272547 18857 ts_tablet_manager.cc:1939] T f936bb575c6347f6bcb315e510dfc204 P ea2db21471f145649b2de067f4df12a4: Deleting consensus metadata
I20260926 19:35:22.274006 18856 ts_tablet_manager.cc:1918] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:35:22.274253 18856 log.cc:1196] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/f936bb575c6347f6bcb315e510dfc204
I20260926 19:35:22.274852 18856 ts_tablet_manager.cc:1939] T f936bb575c6347f6bcb315e510dfc204 P efef9386b20f4b8b92798acdd97da271: Deleting consensus metadata
I20260926 19:35:22.274921 18309 catalog_manager.cc:4841] TS ea2db21471f145649b2de067f4df12a4 (127.6.215.194:36829): tablet f936bb575c6347f6bcb315e510dfc204 (table test-table [id=1901659f47854dc68519b18e6c5bce3d]) successfully deleted
I20260926 19:35:22.275454 18858 ts_tablet_manager.cc:1918] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:35:22.275640 18858 log.cc:1196] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/f936bb575c6347f6bcb315e510dfc204
I20260926 19:35:22.276052 18858 ts_tablet_manager.cc:1939] T f936bb575c6347f6bcb315e510dfc204 P b01c92e64c8a49068bbd8a728c4cdafa: Deleting consensus metadata
I20260926 19:35:22.276612 18307 catalog_manager.cc:4841] TS efef9386b20f4b8b92798acdd97da271 (127.6.215.195:45379): tablet f936bb575c6347f6bcb315e510dfc204 (table test-table [id=1901659f47854dc68519b18e6c5bce3d]) successfully deleted
I20260926 19:35:22.278213 18308 catalog_manager.cc:4841] TS b01c92e64c8a49068bbd8a728c4cdafa (127.6.215.193:46175): tablet f936bb575c6347f6bcb315e510dfc204 (table test-table [id=1901659f47854dc68519b18e6c5bce3d]) successfully deleted
Sep 26 19:35:22 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:35:22.292394 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.254104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.6.215.193:46175)
0926 19:35:22.254327 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.254336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.267867 (+ 13531us) server_negotiation.cc:407] Connection header received
0926 19:35:22.274619 (+ 6752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.274629 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.274730 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.274813 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:22.276753 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.277922 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.280023 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.280413 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.280513 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.285429 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:22.285442 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:22.285456 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:22.285492 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:22.289265 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:22.290031 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:22.290042 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:22.290045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:22.290220 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:22.290625 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:22.290630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:22.290634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:22.291143 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:22.291450 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.291809 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.292030 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:35:22.293390 18531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:35:22.405540 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 18403
I20260926 19:35:22.423338 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 18542
Sep 26 19:35:22 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:35:22.440052 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.424196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.6.215.194:36829)
0926 19:35:22.424360 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.424368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.424409 (+ 41us) server_negotiation.cc:407] Connection header received
0926 19:35:22.424627 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.424633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.424710 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.424850 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:22.426192 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.426944 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.428197 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.428745 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.429052 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.433869 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:22.433886 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:22.433902 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:22.433948 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:22.437126 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:22.437882 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:22.437892 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:22.437896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:22.438047 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:22.438448 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:22.438455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:22.438458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:22.438819 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:22.439123 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.439455 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.439678 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 19:35:22.440687 18674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:35:22.532809 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 18542
I20260926 19:35:22.555740 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 18685
Sep 26 19:35:22 dist-test-slave-phbh krb5kdc[17847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:35:22.573788 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:35:22.556568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.6.215.195:45379)
0926 19:35:22.556806 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 19:35:22.556812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:35:22.556836 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:35:22.557020 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:35:22.557026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:35:22.557119 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:35:22.557236 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:35:22.558534 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.559655 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.561182 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:35:22.561894 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:35:22.562033 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:35:22.566461 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:35:22.566483 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:35:22.566505 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:35:22.566556 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:35:22.570093 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:22.570899 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:22.570907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:22.570909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:22.571034 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:35:22.571571 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:35:22.571578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:35:22.571585 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:35:22.572161 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:35:22.572494 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 19:35:22.573165 (+ 671us) server_negotiation.cc:299] Negotiation successful
0926 19:35:22.573415 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 19:35:22.574644 18808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:35:22.701200 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 18685
I20260926 19:35:22.720345 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 18291
I20260926 19:35:22.721441 18353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:35:22.832679 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 18291
I20260926 19:35:22.854133 7007 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 19:35:23.190667 7007 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 19:35:23.190907 7007 mini_ranger.cc:66] Stopping Ranger...
I20260926 19:35:23.536418 7007 mini_ranger.cc:68] Stopped Ranger
2026-09-26 19:35:23.536 UTC [17877] LOG: received smart shutdown request
2026-09-26 19:35:23.540 UTC [17877] LOG: background worker "logical replication launcher" (PID 17886) exited with exit code 1
2026-09-26 19:35:23.540 UTC [17881] LOG: shutting down
2026-09-26 19:35:23.715 UTC [17877] LOG: database system is shut down
2026-09-26T19:35:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49950 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:35:23 dist-test-slave-phbh krb5kdc[18872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:35:23 dist-test-slave-phbh krb5kdc[18872](info): set up 2 sockets
Sep 26 19:35:23 dist-test-slave-phbh krb5kdc[18872](info): commencing operation
krb5kdc: starting...
W20260926 19:35:24.969213 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:35:25 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:35:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:35:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/rangeradmin_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/rangeradmin_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/rangerlookup_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/rangerlookup_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
I20260926 19:35:25.292248 7007 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taske_ElXL/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/postgres -l logfile start
2026-09-26 19:35:27.846 UTC [18902] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 19:35:27.846 UTC [18902] LOG: listening on IPv4 address "127.6.215.212", port 37683
2026-09-26 19:35:27.850 UTC [18902] LOG: listening on Unix socket "/tmp/.s.PGSQL.37683"
2026-09-26 19:35:27.871 UTC [18905] LOG: database system was shut down at 2026-09-26 19:35:26 UTC
2026-09-26 19:35:27.874 UTC [18902] LOG: database system is ready to accept connections
I20260926 19:35:28.856995 7007 mini_postgres.cc:96] Postgres bound to 37683
2026-09-26 19:35:28.874 UTC [18913] FATAL: database "slave" does not exist
127.6.215.212:37683 - accepting connections
I20260926 19:35:28.874857 7007 mini_ranger.cc:161] Starting Ranger...
I20260926 19:35:28.900887 7007 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 19:35:29.401731 7007 mini_ranger.cc:87] Created ranger Postgres database
I20260926 19:35:29.401847 7007 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-admin
2026-09-26 19:35:29,703 [I] DB FLAVOR :POSTGRES
2026-09-26 19:35:29,703 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 19:35:29,704 [I] Checking connection..
2026-09-26 19:35:29,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 19:35:29,947 [I] Checking connection passed.
2026-09-26 19:35:29,948 [I] --------- Verifying version history table ---------
2026-09-26 19:35:29,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 19:35:30,205 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 19:35:30,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 19:35:30,462 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 19:35:30,463 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 19:35:30,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 19:35:30.693 UTC [18977] WARNING: there is no transaction in progress
2026-09-26 19:35:30,699 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 19:35:30,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 19:35:30,948 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 19:35:30,948 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 19:35:30,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 19:35:31,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 19:35:31,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-phbh.c.gcp-upstream.internal','N') ;"
2026-09-26 19:35:31,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 19:35:31,948 [I] Table x_portal_user does not exist in database ranger
2026-09-26 19:35:31,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 19:35:32,226 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 19:35:32,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 19:35:32,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 19:35:32,724 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 19:35:32,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 19:35:33.696 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.712 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.729 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.744 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.760 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.780 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.845 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.852 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.866 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.876 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.886 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.899 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.909 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.915 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.921 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:33.928 UTC [19085] WARNING: there is no transaction in progress
2026-09-26 19:35:34,567 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 19:35:34,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 19:35:34,807 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 19:35:34,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 19:35:35,061 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 19:35:35,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 19:35:35,290 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 19:35:35,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 19:35:35,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 19:35:35,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-phbh.c.gcp-upstream.internal';"
2026-09-26 19:35:36,036 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 19:35:36,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 19:35:36,296 [I] DB_PATCHES have already been applied
I20260926 19:35:36.301887 7007 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-admin:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taske_ElXL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 19:35:36.301970 7007 mini_ranger.cc:193] Using host: 127.6.215.212
I20260926 19:35:36.317147 7007 mini_ranger.cc:227] Ranger admin URL: http://127.6.215.212:55949
Sep 26, 2026 7:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 7:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 7:35:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 7:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 7:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 19:35:37.459539 7007 mini_ranger.cc:160] Time spent starting Ranger: real 8.585s user 0.004s sys 0.038s
Sep 26, 2026 7:35:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 7:35:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 7:35:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 7:35:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 7:35:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 19:35:57.739655 7007 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.215.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.215.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/rangerkms_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.215.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/rangerkms_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.215.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.215.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.215.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/HTTP_127.6.215.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 19:35:57 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451357, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 19:35:57.903481 7007 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 19:35:57.923156 7007 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 19:35:58.812983 7007 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 19:35:58.813071 7007 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms
I20260926 19:35:58.813102 7007 mini_ranger_kms.cc:227] Using postgres at 127.6.215.212:37683
2026-09-26 19:35:59,150 [I] DB FLAVOR :POSTGRES
2026-09-26 19:35:59,151 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 19:35:59,151 [I] Checking connection
2026-09-26 19:35:59,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 19:35:59,391 [I] connection success
2026-09-26 19:35:59,391 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 19:35:59,392 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 19:35:59,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 19:35:59,669 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 19:35:59,669 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 19:35:59,670 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 19:35:59,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.215.212:37683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 19:35:59,932 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 19:36:00.369987 7007 mini_ranger_kms.cc:310] Created kms service
I20260926 19:36:00.587569 7007 mini_ranger_kms.cc:326] Created kudu user
I20260926 19:36:00.632512 7007 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 19:36:00 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451357, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:00.985539 7007 mini_ranger_kms.cc:384] Added ranger policy
I20260926 19:36:00.985878 7007 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms:/tmp/dist-test-taske_ElXL/build/asan/bin/postgresql.jar:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taske_ElXL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 19:36:00.985947 7007 mini_ranger_kms.cc:240] Using host: 127.6.215.212
I20260926 19:36:01.004257 7007 mini_ranger_kms.cc:276] Ranger KMS PID: 19265
I20260926 19:36:01.004422 7007 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.215.212:45211
Sep 26, 2026 7:36:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 7:36:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 7:36:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 7:36:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 7:36:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 7:36:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 26 19:36:03 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790451363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.215.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:03.405409 7007 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.502s user 0.096s sys 0.255s
I20260926 19:36:03.405655 7007 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 19:36:03.405825 7007 mini_ranger_kms.cc:401] 127.6.215.212:45211/kms/v1/keys
Sep 26 19:36:03 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.215.212@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
Sep 26 19:36:04 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451363, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.215.212@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
Sep 26 19:36:06 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:36:06.313671 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:45853
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:45853
--ranger_config_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:06.654532 19316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:06.654855 19316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:06.654945 19316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:06.662387 19316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:06.662533 19316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:06.662626 19316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:06.662724 19316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:06.662808 19316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:06.673014 19316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.215.212:45211/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:45853
--ranger_config_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:45853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:06.674365 19316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:06.675709 19316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:06.688215 19321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:06.688195 19322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:06.688417 19324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:06.694180 19316 server_base.cc:1034] running on GCE node
I20260926 19:36:06.695106 19316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:06.696632 19316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:06.697837 19316 hybrid_clock.cc:648] HybridClock initialized: now 1790451366697797 us; error 30 us; skew 500 ppm
I20260926 19:36:06.698221 19316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:06 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451366, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:06.701280 19316 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:06.702808 19316 webserver.cc:460] Webserver started at http://127.6.215.254:39229/ using document root <none> and password file <none>
I20260926 19:36:06.703488 19316 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:06.703610 19316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:06.703931 19316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:36:06 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451366, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:06.768652 19316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "984de194cd3143fda6a7bc7b57c2f5ef"
format_stamp: "Formatted at 2026-09-26 19:36:06 on dist-test-slave-phbh"
server_key: "abd6931985cb7ecff6a4837366cf1e3d"
server_key_iv: "a251fa35861b510d9b1a348870cc84e9"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:06.769565 19316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "984de194cd3143fda6a7bc7b57c2f5ef"
format_stamp: "Formatted at 2026-09-26 19:36:06 on dist-test-slave-phbh"
server_key: "abd6931985cb7ecff6a4837366cf1e3d"
server_key_iv: "a251fa35861b510d9b1a348870cc84e9"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:06.775206 19316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:36:06.794898 19331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:06.796507 19316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:36:06.796706 19316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "984de194cd3143fda6a7bc7b57c2f5ef"
format_stamp: "Formatted at 2026-09-26 19:36:06 on dist-test-slave-phbh"
server_key: "abd6931985cb7ecff6a4837366cf1e3d"
server_key_iv: "a251fa35861b510d9b1a348870cc84e9"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:06.796998 19316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:06.829703 19316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:06.833874 19316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:06.834198 19316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:06.853907 19316 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:45853
I20260926 19:36:06.853911 19382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:45853 every 8 connection(s)
I20260926 19:36:06.855320 19316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:06.858675 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 19316
I20260926 19:36:06.858878 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:36:06.859328 19383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:06.859705 7007 external_mini_cluster.cc:1428] Setting key 81fcb933afe154e5dc8ea9594ce53417
I20260926 19:36:06.877885 19383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Bootstrap starting.
I20260926 19:36:06.882200 19383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:06.883530 19383 log.cc:824] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:06.886703 19383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: No bootstrap required, opened a new log
Sep 26 19:36:06 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:06.897320 19383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:06.897825 19383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:06.897939 19383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984de194cd3143fda6a7bc7b57c2f5ef, State: Initialized, Role: FOLLOWER
I20260926 19:36:06.898582 19383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:06.898821 19383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:36:06.898962 19383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:36:06.899156 19383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:06.901522 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:06.861883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.6.215.254:45853)
0926 19:36:06.876871 (+ 14988us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:06.876890 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:06.876942 (+ 52us) server_negotiation.cc:407] Connection header received
0926 19:36:06.879336 (+ 2394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:06.879367 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:06.879953 (+ 586us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:06.880429 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:06.882086 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:06.884161 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:06.885917 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:06.886943 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:06.887132 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:06.892263 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:06.892273 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:06.892296 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:06.892361 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:06.896416 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:06.897338 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:06.897348 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:06.897357 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:06.897518 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:06.897942 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:06.897948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:06.897952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:06.898670 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:06.898976 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:06.899620 (+ 644us) server_negotiation.cc:299] Negotiation successful
0926 19:36:06.900076 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14682,"thread_start_us":14416,"threads_started":1}
I20260926 19:36:06.902458 19383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:06.903017 19383 leader_election.cc:302] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 984de194cd3143fda6a7bc7b57c2f5ef; no voters:
I20260926 19:36:06.903720 19383 leader_election.cc:288] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:36:06.903844 19388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:06.909628 19388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 LEADER]: Becoming Leader. State: Replica: 984de194cd3143fda6a7bc7b57c2f5ef, State: Running, Role: LEADER
I20260926 19:36:06.910277 19388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:06.910588 19383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:06.913456 19390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 984de194cd3143fda6a7bc7b57c2f5ef. Latest consensus state: current_term: 1 leader_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } } }
I20260926 19:36:06.913894 19390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:06.914572 19389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } } }
I20260926 19:36:06.914887 19389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:06.925853 19394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:06.932682 19394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:06.941859 19383 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 19:36:06.956601 19394 catalog_manager.cc:1347] Generated new cluster ID: dd568d3db5ae4314a199b6a19c9db540
I20260926 19:36:06.956756 19394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:36:06.972889 19394 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:36:06.974961 19394 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:36:06.990172 19394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Generated new TSK 0
I20260926 19:36:06.991343 19394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:36:07.773222 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45853
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:36:08 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790451368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 19:36:08.207314 19428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:08.207720 19428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:08.207861 19428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:08.219515 19428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:08.219753 19428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:08.219975 19428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:36:08.235661 19428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.215.212:45211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:08.237437 19428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:08.239185 19428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:08.257112 19438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:08.263566 19442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:08.265343 19439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:08.265319 19428 server_base.cc:1034] running on GCE node
I20260926 19:36:08.266582 19428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:08.269032 19428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:08.270272 19428 hybrid_clock.cc:648] HybridClock initialized: now 1790451368270203 us; error 67 us; skew 500 ppm
I20260926 19:36:08.270650 19428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:08 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:08.273972 19428 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:36:08.275664 19428 webserver.cc:460] Webserver started at http://127.6.215.193:34711/ using document root <none> and password file <none>
I20260926 19:36:08.276427 19428 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:08.276575 19428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:08.276953 19428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:36:08 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:08.297017 19428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "9c3ace1c827a4a7aaea0c47229f18b50"
format_stamp: "Formatted at 2026-09-26 19:36:08 on dist-test-slave-phbh"
server_key: "cf3b9b25a6a65de7f55cc1b99f6ecc0c"
server_key_iv: "05f7f834b4398766d9c7be45a621d9fb"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:08.298578 19428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "9c3ace1c827a4a7aaea0c47229f18b50"
format_stamp: "Formatted at 2026-09-26 19:36:08 on dist-test-slave-phbh"
server_key: "cf3b9b25a6a65de7f55cc1b99f6ecc0c"
server_key_iv: "05f7f834b4398766d9c7be45a621d9fb"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:08.319239 19428 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.007s sys 0.011s
I20260926 19:36:08.341038 19448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:08.350699 19428 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260926 19:36:08.351028 19428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "9c3ace1c827a4a7aaea0c47229f18b50"
format_stamp: "Formatted at 2026-09-26 19:36:08 on dist-test-slave-phbh"
server_key: "cf3b9b25a6a65de7f55cc1b99f6ecc0c"
server_key_iv: "05f7f834b4398766d9c7be45a621d9fb"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:08.351419 19428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:08.411804 19428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:08.417538 19428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:08.417912 19428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:08.419215 19428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:08.420984 19428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:08.421096 19428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:08.421238 19428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:08.421308 19428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 19:36:08 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:08.521986 19428 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:34925
I20260926 19:36:08.523654 19428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:36:08.526073 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 19428
I20260926 19:36:08.526336 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:36:08.527364 7007 external_mini_cluster.cc:1428] Setting key e511b10f8c8c77cddf76eb93b544e626
I20260926 19:36:08.529772 19560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:34925 every 8 connection(s)
Sep 26 19:36:08 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
I20260926 19:36:08.580976 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:08.540372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45853 (local address 127.6.215.193:34951)
0926 19:36:08.541965 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:08.542014 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:08.544470 (+ 2456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:08.545553 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:08.545565 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:08.546190 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:08.546985 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:08.547011 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:08.548591 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:08.548597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:08.550591 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:08.550606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:08.551866 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:08.551873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:08.552171 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:08.554582 (+ 2411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:08.554619 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:08.557237 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:08.565135 (+ 7898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:08.565145 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:08.565166 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:08.565814 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:08.576251 (+ 10437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:08.576257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:08.576261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:08.576906 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:08.577727 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:08.577737 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:08.578255 (+ 518us) client_negotiation.cc:769] Sending connection context
0926 19:36:08.578596 (+ 341us) client_negotiation.cc:240] Negotiation successful
0926 19:36:08.579369 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"thread_start_us":3645,"threads_started":1}
I20260926 19:36:08.582471 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:08.539750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:34951 (local address 127.6.215.254:45853)
0926 19:36:08.541383 (+ 1633us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:08.541392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:08.542133 (+ 741us) server_negotiation.cc:407] Connection header received
0926 19:36:08.544809 (+ 2676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:08.544814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:08.544899 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:08.544981 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:08.547203 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:08.548375 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:08.550771 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:08.551408 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:08.551507 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:08.557526 (+ 6019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:08.557534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:08.557538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:08.557582 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:08.561181 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:08.575796 (+ 14615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:08.575805 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:08.575809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:08.576060 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:08.577083 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:08.577089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:08.577092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:08.577488 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:08.577651 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:08.581009 (+ 3358us) server_negotiation.cc:299] Negotiation successful
0926 19:36:08.582072 (+ 1063us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1498,"thread_start_us":1330,"threads_started":1}
I20260926 19:36:08.584182 19561 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45853
I20260926 19:36:08.584497 19561 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:08.585427 19561 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
I20260926 19:36:08.587977 19348 ts_manager.cc:194] Registered new tserver with Master: 9c3ace1c827a4a7aaea0c47229f18b50 (127.6.215.193:34925)
I20260926 19:36:08.590016 19348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:34951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:36:08.608680 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45853
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:36:08 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
W20260926 19:36:09.064031 19567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:09.064441 19567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:09.064648 19567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:09.076144 19567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:09.076301 19567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:09.076529 19567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:36:09.092103 19567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.215.212:45211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:09.093868 19567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:09.095224 19567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:09.111866 19576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:09.114507 19577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:09.115206 19567 server_base.cc:1034] running on GCE node
W20260926 19:36:09.115751 19580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:09.116549 19567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:09.117621 19567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:09.118858 19567 hybrid_clock.cc:648] HybridClock initialized: now 1790451369118798 us; error 57 us; skew 500 ppm
I20260926 19:36:09.119179 19567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:09.122502 19567 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:36:09.124083 19567 webserver.cc:460] Webserver started at http://127.6.215.194:43977/ using document root <none> and password file <none>
I20260926 19:36:09.124845 19567 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:09.124974 19567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:09.125313 19567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:09.142594 19567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "74527670ed3d4a98b2c1aa620d3abc48"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "cc08be95cca65c2163146d6ad794c6d9"
server_key_iv: "f62f660204020c24750d2ca94b6201c8"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:09.143687 19567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "74527670ed3d4a98b2c1aa620d3abc48"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "cc08be95cca65c2163146d6ad794c6d9"
server_key_iv: "f62f660204020c24750d2ca94b6201c8"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:09.150554 19567 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260926 19:36:09.167289 19591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:09.169162 19567 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 19:36:09.169407 19567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "74527670ed3d4a98b2c1aa620d3abc48"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "cc08be95cca65c2163146d6ad794c6d9"
server_key_iv: "f62f660204020c24750d2ca94b6201c8"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:09.169716 19567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:09.185683 19567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:09.189572 19567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:09.189888 19567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:09.190919 19567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:09.192381 19567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:09.192481 19567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:09.192626 19567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:09.192700 19567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:09.229678 19567 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:40787
I20260926 19:36:09.229701 19703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:40787 every 8 connection(s)
I20260926 19:36:09.230978 19567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:36:09.233410 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 19567
I20260926 19:36:09.233603 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:36:09.234179 7007 external_mini_cluster.cc:1428] Setting key e62294bfe68c760b493e4740fdbeecf3
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:36:09.260002 19707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:09.235396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:51483 (local address 127.6.215.254:45853)
0926 19:36:09.241209 (+ 5813us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:09.241215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:09.241237 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:36:09.241310 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:09.241318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:09.241403 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:09.241521 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:09.243517 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.244730 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.247023 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.247784 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.247934 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:09.252372 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:09.252379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:09.252384 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:09.252426 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:09.256234 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.257276 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.257284 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.257289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.257447 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.257964 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.257972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.257977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.258393 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:09.258554 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:09.259566 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0926 19:36:09.259744 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5674,"thread_start_us":5561,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
I20260926 19:36:09.261180 19706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:09.234608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45853 (local address 127.6.215.194:51483)
0926 19:36:09.235416 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:09.235469 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:09.237593 (+ 2124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:09.241843 (+ 4250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:09.241856 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:09.242454 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:09.243298 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:09.243324 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.244922 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.244927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:09.246851 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:09.246866 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.248046 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.248052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:09.248339 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:09.249168 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:09.249197 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:09.252074 (+ 2877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:09.256405 (+ 4331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:09.256415 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:09.256434 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:09.257132 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:09.257565 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:09.257569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:09.257572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:09.257832 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:09.258546 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:09.258554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:09.258975 (+ 421us) client_negotiation.cc:769] Sending connection context
0926 19:36:09.259271 (+ 296us) client_negotiation.cc:240] Negotiation successful
0926 19:36:09.259798 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":317,"threads_started":1}
I20260926 19:36:09.262748 19704 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45853
I20260926 19:36:09.263079 19704 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:09.263937 19704 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
I20260926 19:36:09.265774 19348 ts_manager.cc:194] Registered new tserver with Master: 74527670ed3d4a98b2c1aa620d3abc48 (127.6.215.194:40787)
I20260926 19:36:09.266995 19348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:51483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:36:09.292600 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45853
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:36:09.593755 19561 heartbeater.cc:503] Master 127.6.215.254:45853 was elected leader, sending a full tablet report...
W20260926 19:36:09.641347 19710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:09.641661 19710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:09.641831 19710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:09.648967 19710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:09.649104 19710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:09.649294 19710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:36:09.659356 19710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.215.212:45211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:09.660620 19710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:09.661942 19710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:09.675032 19715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:09.675253 19718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:09.675379 19716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:09.676007 19710 server_base.cc:1034] running on GCE node
I20260926 19:36:09.676623 19710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:09.677662 19710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:09.678941 19710 hybrid_clock.cc:648] HybridClock initialized: now 1790451369678874 us; error 40 us; skew 500 ppm
I20260926 19:36:09.679277 19710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:09.682556 19710 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:36:09.683974 19710 webserver.cc:460] Webserver started at http://127.6.215.195:39429/ using document root <none> and password file <none>
I20260926 19:36:09.684608 19710 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:09.684705 19710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:09.685022 19710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:09.698783 19710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "dc424fb57471489d97167bae03eb6a92"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "b737c41335885996e3239bce6e131ee2"
server_key_iv: "545aa89f48f3394ac6b886dd9223f4bc"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:09.699698 19710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "dc424fb57471489d97167bae03eb6a92"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "b737c41335885996e3239bce6e131ee2"
server_key_iv: "545aa89f48f3394ac6b886dd9223f4bc"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:09.705606 19710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 19:36:09.722563 19725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:09.724081 19710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:36:09.724255 19710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "dc424fb57471489d97167bae03eb6a92"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "b737c41335885996e3239bce6e131ee2"
server_key_iv: "545aa89f48f3394ac6b886dd9223f4bc"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:09.724537 19710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:09.747468 19710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:09.750989 19710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:09.751241 19710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:09.752092 19710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:09.753454 19710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:09.753549 19710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:09.753633 19710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:09.753676 19710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:09.786175 19710 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:46067
I20260926 19:36:09.786223 19837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:46067 every 8 connection(s)
I20260926 19:36:09.787449 19710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:36:09.791625 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 19710
I20260926 19:36:09.791796 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:36:09.792374 7007 external_mini_cluster.cc:1428] Setting key 9d1dee391fa273bcc909b1e4443934c8
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:09.807523 19840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:09.790027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:43941 (local address 127.6.215.254:45853)
0926 19:36:09.790558 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:09.790562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:09.791501 (+ 939us) server_negotiation.cc:407] Connection header received
0926 19:36:09.793803 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:09.793808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:09.793901 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:09.794012 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:09.795539 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.796254 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.797824 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.798257 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.798390 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:09.801854 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:09.801859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:09.801863 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:09.801895 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:09.804280 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.805066 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.805081 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.805083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.805201 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.805615 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.805619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.805622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.805938 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:09.806090 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:09.806966 (+ 876us) server_negotiation.cc:299] Negotiation successful
0926 19:36:09.807187 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":265,"threads_started":1}
I20260926 19:36:09.808450 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:09.790577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45853 (local address 127.6.215.195:43941)
0926 19:36:09.791296 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:09.791364 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:09.793494 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:09.794254 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:09.794277 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:09.794733 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:09.795380 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:09.795401 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.796397 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.796402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:09.797680 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:09.797692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.798470 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.798478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:09.798711 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:09.799422 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:09.799445 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:09.801628 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:09.804425 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:09.804434 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:09.804452 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:09.804943 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:09.805311 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:09.805316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:09.805320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:09.805491 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:09.806065 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:09.806072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:09.806547 (+ 475us) client_negotiation.cc:769] Sending connection context
0926 19:36:09.806821 (+ 274us) client_negotiation.cc:240] Negotiation successful
0926 19:36:09.807315 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":262,"threads_started":1}
I20260926 19:36:09.809831 19838 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45853
I20260926 19:36:09.810125 19838 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:09.810818 19838 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
I20260926 19:36:09.812242 19348 ts_manager.cc:194] Registered new tserver with Master: dc424fb57471489d97167bae03eb6a92 (127.6.215.195:46067)
I20260926 19:36:09.813084 19348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:43941
I20260926 19:36:09.819836 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:36:09.821503 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 19428
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:36:09.838948 19843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:09.822360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.6.215.193:34925)
0926 19:36:09.822941 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:09.822948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:09.822971 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:36:09.823023 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:09.823030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:09.823221 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:09.823344 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:09.824882 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.825880 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.827404 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.828019 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.828148 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:09.833052 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:09.833064 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:09.833079 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:09.833112 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:09.835922 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.836704 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.836711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.836714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.836920 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.837314 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.837321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.837324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.837657 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:09.837983 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:09.838300 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 19:36:09.838569 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":288,"threads_started":1}
I20260926 19:36:09.840166 19556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:09.923771 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 19428
I20260926 19:36:09.939513 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 19567
Sep 26 19:36:09 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:36:09.954856 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:09.940410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.6.215.194:40787)
0926 19:36:09.940995 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:09.941002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:09.941027 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:36:09.941086 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:09.941092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:09.941307 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:09.941432 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:09.942730 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.943682 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.945093 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:09.945655 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:09.945763 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:09.949926 (+ 4163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:09.949937 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:09.949951 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:09.949989 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:09.952465 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.953176 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.953183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.953185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.953308 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:09.953611 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:09.953614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:09.953616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:09.953929 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:09.954134 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:09.954379 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 19:36:09.954567 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":297,"threads_started":1}
I20260926 19:36:09.956079 19699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:10.036979 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 19567
I20260926 19:36:10.052728 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 19710
Sep 26 19:36:10 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:36:10.069401 19847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:10.053830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.6.215.195:46067)
0926 19:36:10.054432 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:10.054439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:10.054466 (+ 27us) server_negotiation.cc:407] Connection header received
0926 19:36:10.054530 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:10.054536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:10.054736 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:10.054863 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:10.056146 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:10.057331 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:10.058591 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:10.059124 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:10.059225 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:10.063623 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:10.063635 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:10.063651 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:10.063686 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:10.066809 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:10.067549 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:10.067556 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:10.067558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:10.067682 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:10.068003 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:10.068006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:10.068008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:10.068330 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:10.068550 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:10.068904 (+ 354us) server_negotiation.cc:299] Negotiation successful
0926 19:36:10.069108 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":301,"threads_started":1}
I20260926 19:36:10.070693 19833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:10.154057 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 19710
I20260926 19:36:10.169827 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 19316
I20260926 19:36:10.171177 19378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:10.277251 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 19316
I20260926 19:36:10.299925 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:45853
--webserver_interface=127.6.215.254
--webserver_port=39229
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:45853
--ranger_config_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:10.632236 19850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:10.632534 19850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:10.632632 19850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:10.640043 19850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:10.640172 19850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:10.640218 19850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:10.640398 19850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:10.640436 19850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:10.651069 19850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:45853
--ranger_config_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:45853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=39229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:10.652360 19850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:10.653714 19850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:10.666368 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:10.666465 19856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:10.666412 19855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:10.670367 19850 server_base.cc:1034] running on GCE node
I20260926 19:36:10.671216 19850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:10.672706 19850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:10.673966 19850 hybrid_clock.cc:648] HybridClock initialized: now 1790451370673922 us; error 35 us; skew 500 ppm
I20260926 19:36:10.674331 19850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:10 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451370, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:10.677471 19850 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:10.679031 19850 webserver.cc:460] Webserver started at http://127.6.215.254:39229/ using document root <none> and password file <none>
I20260926 19:36:10.679746 19850 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:10.679898 19850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 19:36:10 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451370, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:10.733780 19850 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:36:10.737080 19866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:10.738660 19850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:36:10.738864 19850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "984de194cd3143fda6a7bc7b57c2f5ef"
format_stamp: "Formatted at 2026-09-26 19:36:06 on dist-test-slave-phbh"
server_key: "abd6931985cb7ecff6a4837366cf1e3d"
server_key_iv: "a251fa35861b510d9b1a348870cc84e9"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:10.739856 19850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:10.758605 19850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:10.763200 19850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:10.763521 19850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:10.788316 19850 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:45853
I20260926 19:36:10.788342 19917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:45853 every 8 connection(s)
I20260926 19:36:10.789965 19850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:10.793826 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 19850
I20260926 19:36:10.794687 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:34925
--local_ip_for_outbound_sockets=127.6.215.193
--tserver_master_addrs=127.6.215.254:45853
--webserver_port=34711
--webserver_interface=127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:36:10.796826 19918 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:36:10.799955 19918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Bootstrap starting.
I20260926 19:36:10.818953 19918 log.cc:824] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:10.827702 19918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:36:10.828229 19918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Bootstrap complete.
I20260926 19:36:10.837157 19918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:10.838065 19918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984de194cd3143fda6a7bc7b57c2f5ef, State: Initialized, Role: FOLLOWER
I20260926 19:36:10.838728 19918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:10.838963 19918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:36:10.839082 19918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:36:10.839257 19918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:36:10.843097 19918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:10.843667 19918 leader_election.cc:302] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 984de194cd3143fda6a7bc7b57c2f5ef; no voters:
I20260926 19:36:10.844347 19918 leader_election.cc:288] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 19:36:10.844568 19922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:36:10.846913 19918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:10.849187 19922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [term 2 LEADER]: Becoming Leader. State: Replica: 984de194cd3143fda6a7bc7b57c2f5ef, State: Running, Role: LEADER
I20260926 19:36:10.849890 19922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } }
I20260926 19:36:10.852907 19923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } } }
I20260926 19:36:10.853374 19923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:10.853994 19924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 984de194cd3143fda6a7bc7b57c2f5ef. Latest consensus state: current_term: 2 leader_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984de194cd3143fda6a7bc7b57c2f5ef" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 45853 } } }
I20260926 19:36:10.854329 19924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:10.858103 19931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:10.863665 19931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:10.865195 19931 catalog_manager.cc:1259] Loaded cluster ID: dd568d3db5ae4314a199b6a19c9db540
I20260926 19:36:10.865296 19931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:36:10.869278 19931 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:36:10.870682 19931 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Loaded TSK: 0
I20260926 19:36:10.872195 19931 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:36:10.875609 19918 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260926 19:36:11.250176 19920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:11.250576 19920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:11.250837 19920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:11.262763 19920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:11.263017 19920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:11.263269 19920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:36:11.279109 19920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:34925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=34711
--tserver_master_addrs=127.6.215.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:11.281124 19920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:11.282999 19920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:11.304030 19956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:11.307000 19957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:11.312719 19920 server_base.cc:1034] running on GCE node
W20260926 19:36:11.315011 19960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:11.315922 19920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:11.317433 19920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:11.318738 19920 hybrid_clock.cc:648] HybridClock initialized: now 1790451371318679 us; error 56 us; skew 500 ppm
I20260926 19:36:11.319155 19920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:11 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451371, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:11.323000 19920 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:36:11.330263 19920 webserver.cc:460] Webserver started at http://127.6.215.193:34711/ using document root <none> and password file <none>
I20260926 19:36:11.331028 19920 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:11.331205 19920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 19:36:11 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451371, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:11.360328 19920 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:36:11.365244 19967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:11.373802 19920 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.000s sys 0.003s
I20260926 19:36:11.374022 19920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "9c3ace1c827a4a7aaea0c47229f18b50"
format_stamp: "Formatted at 2026-09-26 19:36:08 on dist-test-slave-phbh"
server_key: "cf3b9b25a6a65de7f55cc1b99f6ecc0c"
server_key_iv: "05f7f834b4398766d9c7be45a621d9fb"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:11.375494 19920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:11.444799 19920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:11.450510 19920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:11.450852 19920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:11.452134 19920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:11.465848 19920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:11.466071 19920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 19:36:11.466275 19920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:11.466409 19920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:11.722559 19920 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:34925
I20260926 19:36:11.724509 19920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:36:11.726572 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 19920
I20260926 19:36:11.727330 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:40787
--local_ip_for_outbound_sockets=127.6.215.194
--tserver_master_addrs=127.6.215.254:45853
--webserver_port=43977
--webserver_interface=127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:36:11.722604 20080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:34925 every 8 connection(s)
Sep 26 19:36:11 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451371, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:11.776634 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:11.734335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45853 (local address 127.6.215.193:45127)
0926 19:36:11.740593 (+ 6258us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:11.740650 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:11.743188 (+ 2538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:11.749416 (+ 6228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:11.749428 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:11.750090 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:11.750862 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:11.750888 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:11.753388 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:11.753394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:11.755466 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:11.755485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:11.757406 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:11.757415 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:11.757694 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:11.762895 (+ 5201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:11.762933 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:11.765793 (+ 2860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:11.771020 (+ 5227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:11.771038 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:11.771057 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:11.771757 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:11.772409 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:11.772414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:11.772419 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:11.772664 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:11.773782 (+ 1118us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:11.773791 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:11.774224 (+ 433us) client_negotiation.cc:769] Sending connection context
0926 19:36:11.774544 (+ 320us) client_negotiation.cc:240] Negotiation successful
0926 19:36:11.775149 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5923,"thread_start_us":5725,"threads_started":1}
I20260926 19:36:11.778317 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:11.734167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:45127 (local address 127.6.215.254:45853)
0926 19:36:11.746564 (+ 12397us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:11.746587 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:11.746644 (+ 57us) server_negotiation.cc:407] Connection header received
0926 19:36:11.747157 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:11.747181 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:11.747725 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:11.748203 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:11.751162 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:11.753145 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:11.755687 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:11.756800 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:11.756994 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:11.766210 (+ 9216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:11.766222 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:11.766245 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:11.766299 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:11.770754 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:11.771984 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:11.772006 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:11.772015 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:11.772249 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:11.772831 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:11.772837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:11.772841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:11.773531 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:11.773697 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:11.776465 (+ 2768us) server_negotiation.cc:299] Negotiation successful
0926 19:36:11.777417 (+ 952us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12088,"thread_start_us":11844,"threads_started":1}
I20260926 19:36:11.782073 20081 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45853
I20260926 19:36:11.782418 20081 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:11.783325 20081 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
W20260926 19:36:11.786652 20081 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:45853 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:36:11.786901 20081 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:11.787212 20081 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
I20260926 19:36:11.798451 20081 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:11.798858 20081 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
W20260926 19:36:11.800113 20081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 19:36:11.925338 20092 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260926 19:36:12.206794 20084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:12.207211 20084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:12.207388 20084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Sep 26 19:36:12 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790451372, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 19:36:12.220387 20084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:12.220557 20084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:12.220803 20084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:36:12.237272 20084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:40787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=43977
--tserver_master_addrs=127.6.215.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:12.239058 20084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:12.240819 20084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:12.261302 20102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:12.261808 20101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:12.266811 20084 server_base.cc:1034] running on GCE node
W20260926 19:36:12.267056 20104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:12.267825 20084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:12.269213 20084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:12.272616 20084 hybrid_clock.cc:648] HybridClock initialized: now 1790451372272545 us; error 59 us; skew 500 ppm
I20260926 19:36:12.273026 20084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:12 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451372, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:12.278033 20084 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:36:12.287990 20084 webserver.cc:460] Webserver started at http://127.6.215.194:43977/ using document root <none> and password file <none>
I20260926 19:36:12.289060 20084 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:12.289224 20084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 19:36:12 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451372, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:12.321712 20084 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 19:36:12.324703 20113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:12.326332 20084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:36:12.326571 20084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "74527670ed3d4a98b2c1aa620d3abc48"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "cc08be95cca65c2163146d6ad794c6d9"
server_key_iv: "f62f660204020c24750d2ca94b6201c8"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:12.328058 20084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:12.345587 20084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:12.351318 20084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:12.351684 20084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:12.353109 20084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:12.355118 20084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:12.355275 20084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 19:36:12.355414 20084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:12.355515 20084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:12.432981 20084 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:40787
I20260926 19:36:12.433037 20225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:40787 every 8 connection(s)
I20260926 19:36:12.435220 20084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:36:12.443879 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20084
I20260926 19:36:12.444964 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.215.212:45211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:46067
--local_ip_for_outbound_sockets=127.6.215.195
--tserver_master_addrs=127.6.215.254:45853
--webserver_port=39429
--webserver_interface=127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:36:12 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451372, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:12.486821 20229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:12.441211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:35777 (local address 127.6.215.254:45853)
0926 19:36:12.461952 (+ 20741us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:12.461960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:12.462621 (+ 661us) server_negotiation.cc:407] Connection header received
0926 19:36:12.465788 (+ 3167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:12.465795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:12.465905 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:12.466015 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:12.469025 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:12.470318 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:12.472799 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:12.473544 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:12.473664 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:12.478889 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:12.478897 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:12.478902 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:12.478949 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:12.482756 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:12.483785 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:12.483792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:12.483796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:12.483954 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:12.484537 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:12.484542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:12.484547 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:12.484975 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:12.485143 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:12.486197 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0926 19:36:12.486439 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20585,"thread_start_us":22291,"threads_started":1}
I20260926 19:36:12.489307 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:12.439681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45853 (local address 127.6.215.194:35777)
0926 19:36:12.462113 (+ 22432us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:12.462191 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:12.465270 (+ 3079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:12.467032 (+ 1762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:12.467047 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:12.467743 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:12.468716 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:12.468747 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:12.470570 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:12.470576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:12.472559 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:12.472588 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:12.474125 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:12.474135 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:12.474462 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:12.475414 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:12.475460 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:12.478566 (+ 3106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:12.482950 (+ 4384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:12.482962 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:12.482994 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:12.483622 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:12.484134 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:12.484139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:12.484143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:12.484391 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:12.485238 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:12.485247 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:12.485720 (+ 473us) client_negotiation.cc:769] Sending connection context
0926 19:36:12.487152 (+ 1432us) client_negotiation.cc:240] Negotiation successful
0926 19:36:12.487750 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21717,"thread_start_us":25930,"threads_started":1}
I20260926 19:36:12.491174 20226 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45853
I20260926 19:36:12.491528 20226 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:12.492482 20226 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
I20260926 19:36:12.494745 19883 ts_manager.cc:194] Registered new tserver with Master: 74527670ed3d4a98b2c1aa620d3abc48 (127.6.215.194:40787)
I20260926 19:36:12.497000 19883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:35777
Sep 26 19:36:12 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451372, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:12.800253 20081 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:12.800724 20081 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
I20260926 19:36:12.801999 19883 ts_manager.cc:194] Registered new tserver with Master: 9c3ace1c827a4a7aaea0c47229f18b50 (127.6.215.193:34925)
I20260926 19:36:12.803186 19883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:45127
W20260926 19:36:12.920434 20230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:12.920864 20230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:12.921027 20230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:12.932339 20230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:12.932502 20230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:12.932745 20230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:36:12.948340 20230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.215.212:45211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:46067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=39429
--tserver_master_addrs=127.6.215.254:45853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:12.949985 20230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:12.951601 20230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:12.988461 20236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:12.992070 20235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:12.994868 20239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:12.995332 20230 server_base.cc:1034] running on GCE node
I20260926 19:36:12.996203 20230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:13.000890 20230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:13.002146 20230 hybrid_clock.cc:648] HybridClock initialized: now 1790451373002050 us; error 99 us; skew 500 ppm
I20260926 19:36:13.002533 20230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:13 dist-test-slave-phbh krb5kdc[18872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451373, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:13.006557 20230 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:36:13.032236 20230 webserver.cc:460] Webserver started at http://127.6.215.195:39429/ using document root <none> and password file <none>
I20260926 19:36:13.033428 20230 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:13.033684 20230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 19:36:13 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451373, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
Sep 26 19:36:13 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451372, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for HTTP/127.6.215.212@KRBTEST.COM
I20260926 19:36:13.084195 20230 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.002s sys 0.007s
I20260926 19:36:13.105461 20247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:13.108291 20230 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.000s sys 0.017s
I20260926 19:36:13.108500 20230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "dc424fb57471489d97167bae03eb6a92"
format_stamp: "Formatted at 2026-09-26 19:36:09 on dist-test-slave-phbh"
server_key: "b737c41335885996e3239bce6e131ee2"
server_key_iv: "545aa89f48f3394ac6b886dd9223f4bc"
server_key_version: "kuduclusterkey@0"
I20260926 19:36:13.109933 20230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:13.150279 20230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:13.155448 20230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:13.155756 20230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:13.178728 20230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:13.193748 20230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:13.193898 20230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:13.194025 20230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:13.194101 20230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:13.349215 20230 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:46067
I20260926 19:36:13.349306 20361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:46067 every 8 connection(s)
I20260926 19:36:13.350919 20230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:36:13.360113 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20230
Sep 26 19:36:13 dist-test-slave-phbh krb5kdc[18872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451373, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:13.377019 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.355229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:33327 (local address 127.6.215.254:45853)
0926 19:36:13.355776 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.355781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.356720 (+ 939us) server_negotiation.cc:407] Connection header received
0926 19:36:13.358614 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.358620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.358724 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.358848 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:13.360687 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.361871 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.364284 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.364987 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.365124 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.369544 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:13.369551 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:13.369555 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:13.369594 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:13.373155 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.374103 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.374109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.374112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.374267 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.374760 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.374765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.374768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.375158 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:13.375302 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.376201 (+ 899us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.376414 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":346,"threads_started":1}
I20260926 19:36:13.378098 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.355801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:45853 (local address 127.6.215.195:33327)
0926 19:36:13.356549 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:13.356594 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:13.358322 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:13.359101 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:13.359110 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:13.359561 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:13.360452 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.360485 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.362076 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.362082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.364084 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.364108 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.365440 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.365447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.365720 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.366484 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:13.366516 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:13.369333 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:13.373296 (+ 3963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:13.373306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:13.373324 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:13.373972 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:13.374400 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:13.374405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:13.374409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:13.374638 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:13.375304 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:13.375312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:13.375749 (+ 437us) client_negotiation.cc:769] Sending connection context
0926 19:36:13.376086 (+ 337us) client_negotiation.cc:240] Negotiation successful
0926 19:36:13.376648 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":309,"threads_started":1}
I20260926 19:36:13.379683 20364 heartbeater.cc:347] Connected to a master server at 127.6.215.254:45853
I20260926 19:36:13.380043 20364 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:13.381076 20364 heartbeater.cc:511] Master 127.6.215.254:45853 requested a full tablet report, sending...
I20260926 19:36:13.382819 19883 ts_manager.cc:194] Registered new tserver with Master: dc424fb57471489d97167bae03eb6a92 (127.6.215.195:46067)
I20260926 19:36:13.384006 19883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:33327
I20260926 19:36:13.387126 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.361288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.6.215.254:45853)
0926 19:36:13.369535 (+ 8247us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.369541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.369558 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:36:13.369607 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.369612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.369706 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.369808 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:13.371914 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.373309 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.375089 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.375791 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.375922 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.380087 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:13.380096 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:13.380100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:13.380146 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:13.383961 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.384863 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.384868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.384871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.385010 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.385529 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.385533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.385536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.385854 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:13.386120 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.386527 (+ 407us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.386783 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7268,"thread_start_us":7114,"threads_started":1}
I20260926 19:36:13.389587 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:36:13.408186 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.392843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.6.215.254:45853)
0926 19:36:13.393081 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.393090 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.393527 (+ 437us) server_negotiation.cc:407] Connection header received
0926 19:36:13.393689 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.393693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.393751 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.393826 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:13.395847 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.396533 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.398567 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.398949 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.399059 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.402271 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:13.402275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:13.402279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:13.402311 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:13.404983 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.405791 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.405794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.405797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.405915 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.406328 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.406330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.406332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.406565 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:13.406810 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.407306 (+ 496us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.407701 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 19:36:13.416484 19883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:36:13.423830 19883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:36:13.456000 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.438387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.6.215.195:46067)
0926 19:36:13.439010 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.439016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.441630 (+ 2614us) server_negotiation.cc:407] Connection header received
0926 19:36:13.441957 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.441963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.442236 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.442429 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:13.443693 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.446570 (+ 2877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.450380 (+ 3810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.452476 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.453246 (+ 770us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.453575 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.455316 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.455556 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":305,"threads_started":1}
I20260926 19:36:13.456255 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.437868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:46067 (local address 127.0.0.1:37204)
0926 19:36:13.441492 (+ 3624us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:13.441531 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:13.441717 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:13.442817 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:13.442828 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:13.442871 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:13.443397 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.443414 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.446784 (+ 3370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.446793 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.450137 (+ 3344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.450158 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.453782 (+ 3624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.453792 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.454092 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.454178 (+ 86us) client_negotiation.cc:769] Sending connection context
0926 19:36:13.454346 (+ 168us) client_negotiation.cc:240] Negotiation successful
0926 19:36:13.454623 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3395,"thread_start_us":3298,"threads_started":1}
I20260926 19:36:13.460596 20295 tablet_service.cc:1467] Processing CreateTablet for tablet a9cc4dc48a534c0796792b70776589fc (DEFAULT_TABLE table=test-table [id=cd1775032a844de1b41fbdef2d8c49a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:13.461515 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.442908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.6.215.194:40787)
0926 19:36:13.453972 (+ 11064us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.453980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.454012 (+ 32us) server_negotiation.cc:407] Connection header received
0926 19:36:13.454132 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.454142 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.454416 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.455374 (+ 958us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:13.455559 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.456872 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.458921 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.459971 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.460119 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.460360 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.461058 (+ 698us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.461224 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10906,"thread_start_us":10777,"threads_started":1}
I20260926 19:36:13.461504 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.442799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:40787 (local address 127.0.0.1:41898)
0926 19:36:13.453065 (+ 10266us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:13.453090 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:13.453228 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:13.454708 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:13.454713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:13.454740 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:13.455146 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.455156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.457032 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.457036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.458780 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.458790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.460530 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.460535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.460737 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.460801 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 19:36:13.460892 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 19:36:13.461051 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10064,"thread_start_us":10531,"threads_started":1}
I20260926 19:36:13.462143 20295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9cc4dc48a534c0796792b70776589fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:13.464752 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.441991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34925 (local address 127.0.0.1:35240)
0926 19:36:13.442571 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:13.442595 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:13.450872 (+ 8277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:13.458069 (+ 7197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:13.458075 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:13.458102 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:13.458414 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.458425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.460243 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.460248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.462325 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.462340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.463809 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.463816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.464000 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.464038 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 19:36:13.464219 (+ 181us) client_negotiation.cc:240] Negotiation successful
0926 19:36:13.464350 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":312,"threads_started":1}
I20260926 19:36:13.464787 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.442339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.6.215.193:34925)
0926 19:36:13.457438 (+ 15099us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.457445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.457468 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:36:13.457595 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.457606 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.457862 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.458052 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:13.458577 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.460068 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.462492 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.463588 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.463758 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.463984 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.464271 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.464422 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14958,"thread_start_us":15888,"threads_started":1}
I20260926 19:36:13.465520 20161 tablet_service.cc:1467] Processing CreateTablet for tablet a9cc4dc48a534c0796792b70776589fc (DEFAULT_TABLE table=test-table [id=cd1775032a844de1b41fbdef2d8c49a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:13.466966 20161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9cc4dc48a534c0796792b70776589fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:13.468001 20016 tablet_service.cc:1467] Processing CreateTablet for tablet a9cc4dc48a534c0796792b70776589fc (DEFAULT_TABLE table=test-table [id=cd1775032a844de1b41fbdef2d8c49a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:13.469293 20016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9cc4dc48a534c0796792b70776589fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:13.477898 20388 tablet_bootstrap.cc:492] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Bootstrap starting.
I20260926 19:36:13.481799 20388 tablet_bootstrap.cc:654] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:13.483398 20388 log.cc:824] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:13.484969 20390 tablet_bootstrap.cc:492] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Bootstrap starting.
I20260926 19:36:13.485850 20389 tablet_bootstrap.cc:492] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Bootstrap starting.
I20260926 19:36:13.487061 20388 tablet_bootstrap.cc:492] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: No bootstrap required, opened a new log
I20260926 19:36:13.487335 20388 ts_tablet_manager.cc:1397] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 19:36:13.489519 20389 tablet_bootstrap.cc:654] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:13.489509 20390 tablet_bootstrap.cc:654] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:13.490782 20390 log.cc:824] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:13.490782 20389 log.cc:824] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:13.494992 20388 raft_consensus.cc:348] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.495350 20388 raft_consensus.cc:374] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:13.495445 20388 raft_consensus.cc:729] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc424fb57471489d97167bae03eb6a92, State: Initialized, Role: FOLLOWER
I20260926 19:36:13.496191 20388 consensus_queue.cc:260] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.497340 20364 heartbeater.cc:503] Master 127.6.215.254:45853 was elected leader, sending a full tablet report...
I20260926 19:36:13.498231 20388 ts_tablet_manager.cc:1428] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260926 19:36:13.498608 20389 tablet_bootstrap.cc:492] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: No bootstrap required, opened a new log
I20260926 19:36:13.498937 20389 ts_tablet_manager.cc:1397] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260926 19:36:13.499037 20390 tablet_bootstrap.cc:492] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: No bootstrap required, opened a new log
I20260926 19:36:13.499276 20390 ts_tablet_manager.cc:1397] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260926 19:36:13.501854 20226 heartbeater.cc:503] Master 127.6.215.254:45853 was elected leader, sending a full tablet report...
I20260926 19:36:13.510015 20390 raft_consensus.cc:348] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.510192 20389 raft_consensus.cc:348] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.510545 20389 raft_consensus.cc:374] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:13.510515 20390 raft_consensus.cc:374] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:13.510630 20389 raft_consensus.cc:729] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74527670ed3d4a98b2c1aa620d3abc48, State: Initialized, Role: FOLLOWER
I20260926 19:36:13.510630 20390 raft_consensus.cc:729] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3ace1c827a4a7aaea0c47229f18b50, State: Initialized, Role: FOLLOWER
I20260926 19:36:13.511264 20390 consensus_queue.cc:260] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.511260 20389 consensus_queue.cc:260] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.512349 20081 heartbeater.cc:503] Master 127.6.215.254:45853 was elected leader, sending a full tablet report...
I20260926 19:36:13.513206 20390 ts_tablet_manager.cc:1428] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260926 19:36:13.513321 20389 ts_tablet_manager.cc:1428] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260926 19:36:13.549549 20394 raft_consensus.cc:482] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:36:13.549835 20394 raft_consensus.cc:504] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.551407 20394 leader_election.cc:288] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c3ace1c827a4a7aaea0c47229f18b50 (127.6.215.193:34925), 74527670ed3d4a98b2c1aa620d3abc48 (127.6.215.194:40787)
I20260926 19:36:13.558001 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.551742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:33129 (local address 127.6.215.193:34925)
0926 19:36:13.551926 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.551931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.551980 (+ 49us) server_negotiation.cc:407] Connection header received
0926 19:36:13.552227 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.552231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.552299 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.552410 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:13.553017 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.554051 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.555951 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.557217 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.557338 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.557391 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.557727 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.557818 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:36:13.558095 20370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.551730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:34925 (local address 127.6.215.195:33129)
0926 19:36:13.551903 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:13.551924 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:13.552058 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:13.552600 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:13.552604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:13.552629 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:13.552860 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.552870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.554191 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.554196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.555773 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.555784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.557363 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.557369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.557547 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.557585 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 19:36:13.557696 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 19:36:13.557829 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":25}
I20260926 19:36:13.558970 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.552318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:44129 (local address 127.6.215.194:40787)
0926 19:36:13.552520 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.552527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.553055 (+ 528us) server_negotiation.cc:407] Connection header received
0926 19:36:13.553212 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.553216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.553272 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.553340 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:13.553864 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.554839 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.557035 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.558147 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.558314 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.558399 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.558716 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.558807 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:36:13.559198 20397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.552140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:40787 (local address 127.6.215.195:44129)
0926 19:36:13.552951 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:13.552970 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:13.553085 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:13.553391 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:13.553396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:13.553419 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:13.553735 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.553745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.554997 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.555001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.556892 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:13.556907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.558431 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.558436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:13.558552 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.558574 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 19:36:13.558681 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 19:36:13.558821 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":3,"thread_start_us":406,"threads_started":1}
I20260926 19:36:13.559180 20036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc4dc48a534c0796792b70776589fc" candidate_uuid: "dc424fb57471489d97167bae03eb6a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" is_pre_election: true
I20260926 19:36:13.559952 20036 raft_consensus.cc:2454] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc424fb57471489d97167bae03eb6a92 in term 0.
I20260926 19:36:13.560258 20181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc4dc48a534c0796792b70776589fc" candidate_uuid: "dc424fb57471489d97167bae03eb6a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74527670ed3d4a98b2c1aa620d3abc48" is_pre_election: true
I20260926 19:36:13.560701 20248 leader_election.cc:302] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c3ace1c827a4a7aaea0c47229f18b50, dc424fb57471489d97167bae03eb6a92; no voters:
I20260926 19:36:13.560859 20181 raft_consensus.cc:2454] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc424fb57471489d97167bae03eb6a92 in term 0.
I20260926 19:36:13.561141 20394 raft_consensus.cc:2781] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:36:13.561307 20394 raft_consensus.cc:482] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:36:13.561447 20394 raft_consensus.cc:3037] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:13.563894 20394 raft_consensus.cc:504] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.564604 20394 leader_election.cc:288] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [CANDIDATE]: Term 1 election: Requested vote from peers 9c3ace1c827a4a7aaea0c47229f18b50 (127.6.215.193:34925), 74527670ed3d4a98b2c1aa620d3abc48 (127.6.215.194:40787)
I20260926 19:36:13.564990 20036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc4dc48a534c0796792b70776589fc" candidate_uuid: "dc424fb57471489d97167bae03eb6a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c3ace1c827a4a7aaea0c47229f18b50"
I20260926 19:36:13.565126 20181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9cc4dc48a534c0796792b70776589fc" candidate_uuid: "dc424fb57471489d97167bae03eb6a92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74527670ed3d4a98b2c1aa620d3abc48"
I20260926 19:36:13.565207 20036 raft_consensus.cc:3037] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:13.565335 20181 raft_consensus.cc:3037] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:13.568357 20181 raft_consensus.cc:2454] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc424fb57471489d97167bae03eb6a92 in term 1.
I20260926 19:36:13.568357 20036 raft_consensus.cc:2454] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc424fb57471489d97167bae03eb6a92 in term 1.
I20260926 19:36:13.568749 20248 leader_election.cc:302] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74527670ed3d4a98b2c1aa620d3abc48, dc424fb57471489d97167bae03eb6a92; no voters:
I20260926 19:36:13.569092 20394 raft_consensus.cc:2781] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:13.569989 20394 raft_consensus.cc:686] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 1 LEADER]: Becoming Leader. State: Replica: dc424fb57471489d97167bae03eb6a92, State: Running, Role: LEADER
I20260926 19:36:13.570551 20394 consensus_queue.cc:237] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } }
I20260926 19:36:13.575129 19883 catalog_manager.cc:5495] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc424fb57471489d97167bae03eb6a92 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "dc424fb57471489d97167bae03eb6a92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc424fb57471489d97167bae03eb6a92" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 46067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 } health_report { overall_health: UNKNOWN } } }
W20260926 19:36:13.604120 20368 tablet.cc:2369] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:36:13.605499 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.600608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.6.215.195:46067)
0926 19:36:13.600855 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.600859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.600875 (+ 16us) server_negotiation.cc:407] Connection header received
0926 19:36:13.600955 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.600958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.601030 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.601151 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:36:13.601480 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.602164 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.603411 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.603819 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.603963 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.604376 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:36:13.605057 (+ 681us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:36:13.605165 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.605261 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.605321 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 19:36:13.612811 20181 raft_consensus.cc:1261] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 1 FOLLOWER]: Refusing update from remote peer dc424fb57471489d97167bae03eb6a92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:13.612815 20036 raft_consensus.cc:1261] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 1 FOLLOWER]: Refusing update from remote peer dc424fb57471489d97167bae03eb6a92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:13.613718 20398 consensus_queue.cc:1035] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74527670ed3d4a98b2c1aa620d3abc48" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 40787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:13.614038 20394 consensus_queue.cc:1035] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3ace1c827a4a7aaea0c47229f18b50" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:13.624714 20403 mvcc.cc:205] Tried to move back new op lower bound from 7333688826308575232 to 7333688826149998592. Current Snapshot: MvccSnapshot[applied={T|T < 7333688826308575232}]
I20260926 19:36:13.642508 19883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:13.642759 19883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:13.646230 19883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 984de194cd3143fda6a7bc7b57c2f5ef: Sending DeleteTablet for 3 replicas of tablet a9cc4dc48a534c0796792b70776589fc
I20260926 19:36:13.647311 20295 tablet_service.cc:1514] Processing DeleteTablet for tablet a9cc4dc48a534c0796792b70776589fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:13 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:37204
I20260926 19:36:13.647379 20016 tablet_service.cc:1514] Processing DeleteTablet for tablet a9cc4dc48a534c0796792b70776589fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:13 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:35240
I20260926 19:36:13.647500 20161 tablet_service.cc:1514] Processing DeleteTablet for tablet a9cc4dc48a534c0796792b70776589fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:13 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:41898
I20260926 19:36:13.648447 20412 tablet_replica.cc:330] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: stopping tablet replica
I20260926 19:36:13.648892 20412 raft_consensus.cc:2229] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:13.649362 20412 raft_consensus.cc:2258] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:13.650123 20413 tablet_replica.cc:330] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: stopping tablet replica
I20260926 19:36:13.650731 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 19920
I20260926 19:36:13.650511 20413 raft_consensus.cc:2229] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:13.651556 20413 raft_consensus.cc:2258] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:13.653903 20413 ts_tablet_manager.cc:1905] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:13.660925 20413 ts_tablet_manager.cc:1918] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:13.661108 20413 log.cc:1196] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/a9cc4dc48a534c0796792b70776589fc
I20260926 19:36:13.661628 20413 ts_tablet_manager.cc:1939] T a9cc4dc48a534c0796792b70776589fc P 74527670ed3d4a98b2c1aa620d3abc48: Deleting consensus metadata
I20260926 19:36:13.663419 19867 catalog_manager.cc:4841] TS 74527670ed3d4a98b2c1aa620d3abc48 (127.6.215.194:40787): tablet a9cc4dc48a534c0796792b70776589fc (table test-table [id=cd1775032a844de1b41fbdef2d8c49a8]) successfully deleted
I20260926 19:36:13.663834 20411 tablet_replica.cc:330] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: stopping tablet replica
I20260926 19:36:13.664209 20412 ts_tablet_manager.cc:1905] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:13.664253 20411 raft_consensus.cc:2229] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:36:13.664886 20411 raft_consensus.cc:2258] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:13.666730 20411 ts_tablet_manager.cc:1905] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:13.671172 20412 ts_tablet_manager.cc:1918] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:13.671345 20412 log.cc:1196] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/a9cc4dc48a534c0796792b70776589fc
I20260926 19:36:13.671893 20412 ts_tablet_manager.cc:1939] T a9cc4dc48a534c0796792b70776589fc P 9c3ace1c827a4a7aaea0c47229f18b50: Deleting consensus metadata
I20260926 19:36:13.673718 19867 catalog_manager.cc:4841] TS 9c3ace1c827a4a7aaea0c47229f18b50 (127.6.215.193:34925): tablet a9cc4dc48a534c0796792b70776589fc (table test-table [id=cd1775032a844de1b41fbdef2d8c49a8]) successfully deleted
I20260926 19:36:13.673846 20411 ts_tablet_manager.cc:1918] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:13.674016 20411 log.cc:1196] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/a9cc4dc48a534c0796792b70776589fc
I20260926 19:36:13.674582 20411 ts_tablet_manager.cc:1939] T a9cc4dc48a534c0796792b70776589fc P dc424fb57471489d97167bae03eb6a92: Deleting consensus metadata
I20260926 19:36:13.675496 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.652523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.6.215.193:34925)
0926 19:36:13.652689 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.652695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.652715 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:36:13.652798 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.652803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.652888 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.652986 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:13.659724 (+ 6738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.660823 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.663369 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.664113 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.664267 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.668239 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:13.668250 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:13.668263 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:13.668304 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:13.672308 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.673140 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.673150 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.673154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.673310 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.673770 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.673775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.673778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.674256 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:13.674563 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.674895 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.675124 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260926 19:36:13.676386 20076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:13.676483 19867 catalog_manager.cc:4841] TS dc424fb57471489d97167bae03eb6a92 (127.6.215.195:46067): tablet a9cc4dc48a534c0796792b70776589fc (table test-table [id=cd1775032a844de1b41fbdef2d8c49a8]) successfully deleted
I20260926 19:36:13.766072 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 19920
I20260926 19:36:13.789191 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20084
I20260926 19:36:13.806562 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.790067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.6.215.194:40787)
0926 19:36:13.790236 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.790242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.790262 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:36:13.790451 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.790457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.790531 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.790630 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:13.792319 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.793432 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.795324 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.795972 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.796099 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.799427 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:13.799445 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:13.799465 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:13.799508 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:13.803492 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.804344 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.804354 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.804358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.804509 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.804932 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.804936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.804939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.805375 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:13.805686 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.805964 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.806175 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:36:13.807288 20221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:13.896873 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20084
I20260926 19:36:13.914206 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20230
I20260926 19:36:13.928387 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:13.914990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37224 (local address 127.6.215.195:46067)
0926 19:36:13.915242 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:13.915249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:13.915269 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:36:13.915469 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:13.915475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:13.915552 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:13.915670 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:13.917100 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.917903 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.919223 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:13.919681 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:13.919833 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:13.922665 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:13.922678 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:13.922698 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:13.922737 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:13.925593 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.926245 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.926253 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.926255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.926414 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:13.926835 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:13.926841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:13.926851 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:13.927183 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:13.927484 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:13.927786 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 19:36:13.927995 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260926 19:36:13.929066 20357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:14.050710 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20230
I20260926 19:36:14.070565 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 19850
I20260926 19:36:14.071784 19913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:14.182811 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 19850
I20260926 19:36:14.207242 7007 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 19:36:14.542680 7007 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 19:36:14.542909 7007 mini_ranger.cc:66] Stopping Ranger...
I20260926 19:36:14.888227 7007 mini_ranger.cc:68] Stopped Ranger
2026-09-26 19:36:14.888 UTC [18902] LOG: received smart shutdown request
2026-09-26 19:36:14.891 UTC [18902] LOG: background worker "logical replication launcher" (PID 18911) exited with exit code 1
2026-09-26 19:36:14.891 UTC [18906] LOG: shutting down
2026-09-26 19:36:15.060 UTC [18902] LOG: database system is shut down
2026-09-26T19:36:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51352 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:36:15 dist-test-slave-phbh krb5kdc[20426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:36:15 dist-test-slave-phbh krb5kdc[20426](info): set up 2 sockets
Sep 26 19:36:15 dist-test-slave-phbh krb5kdc[20426](info): commencing operation
krb5kdc: starting...
W20260926 19:36:16.321311 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:36:16 dist-test-slave-phbh krb5kdc[20426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:36:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:36:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:36:16.553673 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:36285
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:36285,127.6.215.253:37343,127.6.215.252:45455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:16.880340 20439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:16.880638 20439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:16.880868 20439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:16.888181 20439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:16.888314 20439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:16.888403 20439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:16.888515 20439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:16.888598 20439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:16.898730 20439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:36285,127.6.215.253:37343,127.6.215.252:45455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:16.900074 20439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:16.901461 20439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:16.913318 20444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:16.913408 20447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:16.913378 20445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:16.913853 20439 server_base.cc:1034] running on GCE node
I20260926 19:36:16.914703 20439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:16.916049 20439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:16.917279 20439 hybrid_clock.cc:648] HybridClock initialized: now 1790451376917241 us; error 48 us; skew 500 ppm
I20260926 19:36:16.917659 20439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:16 dist-test-slave-phbh krb5kdc[20426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:16.925814 20439 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:16.927392 20439 webserver.cc:460] Webserver started at http://127.6.215.254:43795/ using document root <none> and password file <none>
I20260926 19:36:16.928102 20439 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:16.928251 20439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:16.928565 20439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:16.931579 20439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "ace1946523b645539e4c501317048993"
format_stamp: "Formatted at 2026-09-26 19:36:16 on dist-test-slave-phbh"
server_key: "1ca7e737ba9a44edf5c743d66520e811"
server_key_iv: "2bcbbf42c99d89923817aa5cde34ed1b"
server_key_version: "encryptionkey@0"
I20260926 19:36:16.932394 20439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "ace1946523b645539e4c501317048993"
format_stamp: "Formatted at 2026-09-26 19:36:16 on dist-test-slave-phbh"
server_key: "1ca7e737ba9a44edf5c743d66520e811"
server_key_iv: "2bcbbf42c99d89923817aa5cde34ed1b"
server_key_version: "encryptionkey@0"
I20260926 19:36:16.937654 20439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 19:36:16.941223 20454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:16.942636 20439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:36:16.942813 20439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "ace1946523b645539e4c501317048993"
format_stamp: "Formatted at 2026-09-26 19:36:16 on dist-test-slave-phbh"
server_key: "1ca7e737ba9a44edf5c743d66520e811"
server_key_iv: "2bcbbf42c99d89923817aa5cde34ed1b"
server_key_version: "encryptionkey@0"
I20260926 19:36:16.943015 20439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:16.958886 20439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:16.963315 20439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:16.963605 20439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:16.982293 20439 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:36285
I20260926 19:36:16.982316 20505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:36285 every 8 connection(s)
I20260926 19:36:16.983690 20439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:16.985610 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20439
I20260926 19:36:16.985845 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:36:16.986693 7007 external_mini_cluster.cc:1428] Setting key 368dcd1d90b06ec7dfed69fc4f0ac23b
I20260926 19:36:16.988237 20506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:16.997267 20506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.215.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.253@KRBTEST.COM" created.
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:17.019546 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.000466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36285 (local address 127.0.0.1:51636)
0926 19:36:17.001273 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.001316 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:17.003317 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:17.004109 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:17.004117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:17.004856 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:17.005194 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.005211 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.006733 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.006741 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.007795 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.007806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.008723 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.008730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.008991 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.009663 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:17.009684 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:17.012070 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:17.015294 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.015307 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.015325 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.015714 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.016122 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.016130 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.016143 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.016292 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.017144 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:17.017152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:17.017348 (+ 196us) client_negotiation.cc:769] Sending connection context
0926 19:36:17.017584 (+ 236us) client_negotiation.cc:240] Negotiation successful
0926 19:36:17.018153 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":180,"thread_start_us":349,"threads_started":1}
I20260926 19:36:17.019506 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.001861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51636 (local address 127.6.215.254:36285)
0926 19:36:17.002522 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.002533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.002586 (+ 53us) server_negotiation.cc:407] Connection header received
0926 19:36:17.003682 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.003687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.003899 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.004097 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:17.005386 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.006574 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.007934 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.008463 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.008678 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.012296 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:17.012304 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:17.012315 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:17.012343 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:17.015158 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.015845 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.015855 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.015863 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.016014 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.016402 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.016414 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.016417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.017006 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:17.017210 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:17.017703 (+ 493us) server_negotiation.cc:299] Negotiation successful
0926 19:36:17.018038 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":483,"thread_start_us":316,"threads_started":1}
I20260926 19:36:17.023156 20506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:36:17.024340 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.023833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54682)
0926 19:36:17.024038 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.024181 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260926 19:36:17.024653 20457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
I20260926 19:36:17.026399 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.025932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54684)
0926 19:36:17.026122 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.026217 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 19:36:17.026623 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:37343: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab.
Entry for principal kudu/127.6.215.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab.
I20260926 19:36:17.038311 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:37343
--webserver_interface=127.6.215.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab
--principal=kudu/127.6.215.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:36285,127.6.215.253:37343,127.6.215.252:45455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:36:17.075968 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } attempt: 1
I20260926 19:36:17.077279 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.076798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54686)
0926 19:36:17.077010 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.077142 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 19:36:17.078361 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.078002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54688)
0926 19:36:17.078153 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.078231 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 19:36:17.078554 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:37343: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
I20260926 19:36:17.146790 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } attempt: 2
I20260926 19:36:17.148061 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.147523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54690)
0926 19:36:17.147812 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.147912 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 19:36:17.149092 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.148689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54694)
0926 19:36:17.148872 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.148969 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 19:36:17.149279 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:37343: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
I20260926 19:36:17.240549 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } attempt: 3
I20260926 19:36:17.242074 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.241331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54698)
0926 19:36:17.241687 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.241845 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260926 19:36:17.243345 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.242808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54702)
0926 19:36:17.243018 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.243138 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 19:36:17.243644 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:37343: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
W20260926 19:36:17.366056 20513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:17.366348 20513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:17.366541 20513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:17.373943 20513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:17.374058 20513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:17.374145 20513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:17.374253 20513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:17.374330 20513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:17.384593 20513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:36285,127.6.215.253:37343,127.6.215.252:45455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab
--principal=kudu/127.6.215.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:37343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.215.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:17.385967 20513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:17.386925 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } attempt: 4
I20260926 19:36:17.387492 20513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 19:36:17.388242 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.387639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54710)
0926 19:36:17.387958 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.388078 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260926 19:36:17.389420 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.389002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54720)
0926 19:36:17.389159 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.389260 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 19:36:17.389638 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:37343: Network error: Client connection negotiation failed: client connection to 127.6.215.253:37343: connect: Connection refused (error 111)
W20260926 19:36:17.400306 20518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:17.400398 20521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:17.400362 20519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:17.401502 20513 server_base.cc:1034] running on GCE node
I20260926 19:36:17.402400 20513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:17.403970 20513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:17.405229 20513 hybrid_clock.cc:648] HybridClock initialized: now 1790451377405181 us; error 49 us; skew 500 ppm
I20260926 19:36:17.405671 20513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:17.410014 20513 init.cc:377] Logged in from keytab as kudu/127.6.215.253@KRBTEST.COM (short username kudu)
I20260926 19:36:17.411851 20513 webserver.cc:460] Webserver started at http://127.6.215.253:45495/ using document root <none> and password file <none>
I20260926 19:36:17.412842 20513 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:17.412983 20513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:17.413347 20513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:17.418018 20513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data/instance:
uuid: "3b21a405716b4fbba161634d4d284e6b"
format_stamp: "Formatted at 2026-09-26 19:36:17 on dist-test-slave-phbh"
server_key: "5d76391a5eb09c3a68126ece96ba29bc"
server_key_iv: "09f139c2f8bf61015594ebad386ce95c"
server_key_version: "encryptionkey@0"
I20260926 19:36:17.418900 20513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/wal/instance:
uuid: "3b21a405716b4fbba161634d4d284e6b"
format_stamp: "Formatted at 2026-09-26 19:36:17 on dist-test-slave-phbh"
server_key: "5d76391a5eb09c3a68126ece96ba29bc"
server_key_iv: "09f139c2f8bf61015594ebad386ce95c"
server_key_version: "encryptionkey@0"
I20260926 19:36:17.424638 20513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:36:17.428609 20528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:17.430267 20513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:17.430459 20513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/wal
uuid: "3b21a405716b4fbba161634d4d284e6b"
format_stamp: "Formatted at 2026-09-26 19:36:17 on dist-test-slave-phbh"
server_key: "5d76391a5eb09c3a68126ece96ba29bc"
server_key_iv: "09f139c2f8bf61015594ebad386ce95c"
server_key_version: "encryptionkey@0"
I20260926 19:36:17.430670 20513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:17.447713 20513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:17.452342 20513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:17.452666 20513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:17.471370 20513 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.253:37343
I20260926 19:36:17.471391 20579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.253:37343 every 8 connection(s)
I20260926 19:36:17.472868 20513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
I20260926 19:36:17.476465 20580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:17.480587 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20513
I20260926 19:36:17.480850 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-1/wal/instance
I20260926 19:36:17.481412 7007 external_mini_cluster.cc:1428] Setting key 775c1330749ab610423844e4bc900396
I20260926 19:36:17.485008 20580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.252@KRBTEST.COM" created.
I20260926 19:36:17.506934 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.488695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.6.215.254:36285)
0926 19:36:17.488981 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.488991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.490176 (+ 1185us) server_negotiation.cc:407] Connection header received
0926 19:36:17.492121 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.492127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.492228 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.492346 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:17.493998 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.494690 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.496302 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.496749 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.496945 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.500914 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:17.500920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:17.500923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:17.500958 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:17.503668 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.504393 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.504397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.504399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.504554 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.505032 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.505038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.505041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.505377 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:17.505535 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:17.506217 (+ 682us) server_negotiation.cc:299] Negotiation successful
0926 19:36:17.506475 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260926 19:36:17.507526 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.489202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36285 (local address 127.0.0.1:51644)
0926 19:36:17.489996 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.490036 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:17.491831 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:17.492592 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:17.492600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:17.493222 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:17.493817 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.493836 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.494868 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.494872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.496144 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.496156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.497038 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.497045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.497293 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.498033 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:17.498056 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:17.500644 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:17.503831 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.503838 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.503850 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.504286 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.504672 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.504675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.504677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.504914 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.505508 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:17.505514 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:17.505831 (+ 317us) client_negotiation.cc:769] Sending connection context
0926 19:36:17.506065 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 19:36:17.506433 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":374,"threads_started":1}
I20260926 19:36:17.509377 20580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:36:17.524389 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.510221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54730)
0926 19:36:17.510402 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.510422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:17.510591 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:17.511283 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:17.511287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:17.511723 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:17.511845 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.511853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.513797 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.513803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.514640 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.514651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.515461 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.515466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.515684 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.516262 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:17.516280 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:17.518562 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:17.521929 (+ 3367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.521937 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.521942 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.522299 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.522673 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.522679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.522683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.522832 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.523406 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:17.523411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:17.523552 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 19:36:17.523700 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 19:36:17.523990 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 19:36:17.524466 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.510294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54730 (local address 127.6.215.253:37343)
0926 19:36:17.510887 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.510893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.510915 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:36:17.510952 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.510957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.511134 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.511298 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:17.511993 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.513671 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.514775 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.515310 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.515440 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.518772 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:17.518783 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:17.518797 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:17.518824 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:17.521794 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.522429 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.522438 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.522443 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.522564 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.522933 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.522941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.522944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.523285 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:17.523493 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:17.523845 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 19:36:17.524102 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":447,"thread_start_us":294,"threads_started":1}
I20260926 19:36:17.526913 20580 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:36:17.528182 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.527553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48828)
0926 19:36:17.527795 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.527991 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260926 19:36:17.528417 20531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
I20260926 19:36:17.530246 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.529884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48844)
0926 19:36:17.530043 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.530129 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 19:36:17.530543 20580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab.
Entry for principal kudu/127.6.215.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab.
I20260926 19:36:17.534148 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:45455
--webserver_interface=127.6.215.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab
--principal=kudu/127.6.215.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:36285,127.6.215.253:37343,127.6.215.252:45455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:36:17.579882 20580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 1
I20260926 19:36:17.581235 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.580690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48852)
0926 19:36:17.580966 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.581082 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260926 19:36:17.582468 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.582118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48858)
0926 19:36:17.582272 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.582342 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 19:36:17.582739 20580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
I20260926 19:36:17.650966 20580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 2
I20260926 19:36:17.652150 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.651695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48868)
0926 19:36:17.651882 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.651973 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 19:36:17.653173 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.652863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48872)
0926 19:36:17.652983 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.653067 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 19:36:17.653396 20580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
I20260926 19:36:17.688946 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } attempt: 5
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:36:17.702819 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.690009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54744 (local address 127.6.215.253:37343)
0926 19:36:17.690228 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.690235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.690256 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:36:17.690324 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.690327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.690402 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.690512 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:17.691309 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.692013 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.693229 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.693700 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.693838 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.697359 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:17.697365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:17.697368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:17.697400 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:17.700005 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.700887 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.700892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.700894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.701023 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.701464 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.701469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.701472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.701781 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:17.701894 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:17.702415 (+ 521us) server_negotiation.cc:299] Negotiation successful
0926 19:36:17.702597 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 19:36:17.702973 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.689774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54744)
0926 19:36:17.689983 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.690011 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:17.690164 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:17.690568 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:17.690573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:17.690970 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:17.691149 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.691157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.692158 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.692166 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.693080 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.693093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.693868 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.693874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.694058 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.694708 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:17.694726 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:17.697153 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:17.700160 (+ 3007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.700166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.700170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.700674 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.701145 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.701150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.701154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.701339 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.701914 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:17.701921 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:17.702134 (+ 213us) client_negotiation.cc:769] Sending connection context
0926 19:36:17.702321 (+ 187us) client_negotiation.cc:240] Negotiation successful
0926 19:36:17.702648 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 19:36:17.704092 20506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:36:17.705190 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.704821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48880)
0926 19:36:17.704979 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.705051 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 19:36:17.706185 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.705867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48884)
0926 19:36:17.706005 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.706090 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 19:36:17.706463 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
I20260926 19:36:17.744616 20580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 3
I20260926 19:36:17.745816 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.745343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48896)
0926 19:36:17.745566 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.745663 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 19:36:17.746940 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.746519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48910)
0926 19:36:17.746748 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.746829 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260926 19:36:17.747160 20580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
I20260926 19:36:17.757695 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 1
I20260926 19:36:17.758770 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.758334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48914)
0926 19:36:17.758516 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.758621 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 19:36:17.759765 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.759441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48928)
0926 19:36:17.759585 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.759662 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 19:36:17.759974 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
I20260926 19:36:17.828212 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 2
I20260926 19:36:17.829423 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.828942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48934)
0926 19:36:17.829177 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.829264 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 19:36:17.830467 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.830117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48944)
0926 19:36:17.830269 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.830342 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 19:36:17.830693 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
W20260926 19:36:17.862264 20587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:17.862715 20587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:17.862891 20587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:17.870170 20587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:17.870333 20587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:17.870447 20587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:17.870555 20587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:17.870637 20587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:17.880492 20587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:36285,127.6.215.253:37343,127.6.215.252:45455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab
--principal=kudu/127.6.215.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:45455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.215.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:17.881918 20587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:17.883311 20587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 19:36:17.890384 20580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 4
I20260926 19:36:17.891566 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.891068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48954)
0926 19:36:17.891310 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.891394 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 19:36:17.892620 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.892222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48968)
0926 19:36:17.892380 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.892484 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 19:36:17.892946 20580 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
W20260926 19:36:17.895751 20592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:17.895692 20595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:17.895816 20593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:17.896469 20587 server_base.cc:1034] running on GCE node
I20260926 19:36:17.897267 20587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:17.898562 20587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:17.899775 20587 hybrid_clock.cc:648] HybridClock initialized: now 1790451377899728 us; error 41 us; skew 500 ppm
I20260926 19:36:17.900138 20587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:17.903326 20587 init.cc:377] Logged in from keytab as kudu/127.6.215.252@KRBTEST.COM (short username kudu)
I20260926 19:36:17.904886 20587 webserver.cc:460] Webserver started at http://127.6.215.252:42781/ using document root <none> and password file <none>
I20260926 19:36:17.905575 20587 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:17.905718 20587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:17.906014 20587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:17.909017 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data/instance:
uuid: "ba47a1c6672847688001aa37186ec56c"
format_stamp: "Formatted at 2026-09-26 19:36:17 on dist-test-slave-phbh"
server_key: "f0e84e5eed9f032f038035251df5256b"
server_key_iv: "66b7d5239fc62050c75b742e93bf1587"
server_key_version: "encryptionkey@0"
I20260926 19:36:17.909910 20587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/wal/instance:
uuid: "ba47a1c6672847688001aa37186ec56c"
format_stamp: "Formatted at 2026-09-26 19:36:17 on dist-test-slave-phbh"
server_key: "f0e84e5eed9f032f038035251df5256b"
server_key_iv: "66b7d5239fc62050c75b742e93bf1587"
server_key_version: "encryptionkey@0"
I20260926 19:36:17.915055 20587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:36:17.918455 20602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:17.919838 20587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:36:17.920014 20587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/wal
uuid: "ba47a1c6672847688001aa37186ec56c"
format_stamp: "Formatted at 2026-09-26 19:36:17 on dist-test-slave-phbh"
server_key: "f0e84e5eed9f032f038035251df5256b"
server_key_iv: "66b7d5239fc62050c75b742e93bf1587"
server_key_version: "encryptionkey@0"
I20260926 19:36:17.920231 20587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:17.936865 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 3
I20260926 19:36:17.937889 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.937471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48982)
0926 19:36:17.937648 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.937732 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 19:36:17.938841 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.938493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:48998)
0926 19:36:17.938677 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.938741 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 19:36:17.939040 20506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:45455: Network error: Client connection negotiation failed: client connection to 127.6.215.252:45455: connect: Connection refused (error 111)
I20260926 19:36:17.939131 20587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:17.943691 20587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:17.943967 20587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:17.962678 20587 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.252:45455
I20260926 19:36:17.962708 20653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.252:45455 every 8 connection(s)
I20260926 19:36:17.964066 20587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
I20260926 19:36:17.966274 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20587
I20260926 19:36:17.966464 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790451176367082-7007-0/minicluster-data/master-2/wal/instance
I20260926 19:36:17.967023 7007 external_mini_cluster.cc:1428] Setting key dac26474c7b5290529aa1f0f37df0f41
I20260926 19:36:17.968323 20654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:17.977550 20654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:17.991065 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.968923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.6.215.252:45455)
0926 19:36:17.969721 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.969732 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.969770 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:36:17.971278 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.971300 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.971700 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.972028 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:17.973912 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.975279 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.977226 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.977895 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.978028 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.982874 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:17.982884 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:17.982897 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:17.982950 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:17.986476 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.987327 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.987335 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.987343 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.987600 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.988054 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.988060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.988063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.988747 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:17.989077 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:17.989547 (+ 470us) server_negotiation.cc:299] Negotiation successful
0926 19:36:17.989835 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"spinlock_wait_cycles":16768,"thread_start_us":371,"threads_started":1}
I20260926 19:36:17.996906 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.980638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.6.215.254:36285)
0926 19:36:17.980881 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.980888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.981481 (+ 593us) server_negotiation.cc:407] Connection header received
0926 19:36:17.981773 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.981784 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.981869 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.981931 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:17.983340 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.984398 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.985898 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.986345 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.986476 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.990247 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:17.990255 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:17.990260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:17.990300 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:17.993994 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.994898 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.994906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.994912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.995063 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:17.995423 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:17.995429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:17.995433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:17.995799 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:17.995966 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:17.996377 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 19:36:17.996565 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 19:36:17.996992 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.980508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:36285 (local address 127.0.0.1:51652)
0926 19:36:17.981353 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.981400 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:17.981611 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:17.982152 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:17.982156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:17.982754 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:17.983165 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.983178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.984567 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.984575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.985750 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:17.985762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.986536 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.986543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:17.986722 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:17.987327 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:17.987348 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:17.990043 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:17.994159 (+ 4116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.994173 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.994183 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.994756 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.995181 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:17.995184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:17.995186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:17.995296 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:17.995934 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:17.995940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:17.996078 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 19:36:17.996212 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 19:36:17.996524 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":463,"threads_started":1}
I20260926 19:36:17.998377 20654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:36:17 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
Sep 26 19:36:18 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:36:18.010435 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.994014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.6.215.254:36285)
0926 19:36:17.994597 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.994601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.994620 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:36:17.994657 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.994660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.994709 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.994825 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:17.996426 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:17.997702 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:17.999762 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.000168 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.000304 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.004300 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.004309 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.004317 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.004359 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.007785 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.008432 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.008437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.008440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.008580 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.009005 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.009010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.009013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.009299 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.009529 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.009861 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.010021 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":276,"threads_started":1}
I20260926 19:36:18.011469 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.999100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:37343 (local address 127.0.0.1:54752)
0926 19:36:17.999274 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:17.999292 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:17.999381 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:17.999686 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:17.999689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:18.000005 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:18.000138 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.000147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.001160 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.001168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.002017 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.002028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.002748 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.002754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.002950 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.003502 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:18.003527 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:18.005520 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:18.008986 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.008994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.008999 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.009392 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.009814 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.009819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.009822 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.009993 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.010697 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:18.010702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:18.010878 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 19:36:18.011032 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 19:36:18.011254 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 19:36:18.011922 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:17.999137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.6.215.253:37343)
0926 19:36:17.999327 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:17.999333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:17.999352 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:36:17.999512 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:17.999516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:17.999575 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:17.999654 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.000295 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.001025 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.002157 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.002583 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.002734 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.005720 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.005728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.005733 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.005776 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.008857 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.009518 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.009524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.009527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.009678 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.010120 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.010125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.010128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.010437 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.010596 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.011155 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.011376 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:36:18.012902 20654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:36:18 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
Sep 26 19:36:18 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:36:18.027680 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.013551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:49026)
0926 19:36:18.013711 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:18.013732 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:18.013830 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:18.014113 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:18.014119 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:18.014599 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:18.014737 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.014745 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.015986 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.015991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.017348 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.017362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.018310 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.018315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.018503 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.019191 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:18.019216 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:18.021843 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:18.025395 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.025400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.025404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.025735 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.026083 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.026089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.026092 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.026217 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.026671 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:18.026675 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:18.026811 (+ 136us) client_negotiation.cc:769] Sending connection context
0926 19:36:18.026957 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 19:36:18.027253 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 19:36:18.027711 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.013620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.6.215.252:45455)
0926 19:36:18.013780 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:18.013784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:18.013798 (+ 14us) server_negotiation.cc:407] Connection header received
0926 19:36:18.013934 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:18.013938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:18.013986 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:18.014056 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.014865 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.015868 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.017487 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.018143 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.018269 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.022002 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.022007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.022010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.022037 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.025270 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.025846 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.025853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.025856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.025980 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.026315 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.026322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.026325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.026558 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.026701 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.027028 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.027398 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":75}
I20260926 19:36:18.028569 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.012334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54758 (local address 127.6.215.253:37343)
0926 19:36:18.012483 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:18.012488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:18.012510 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:36:18.012722 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:18.012729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:18.012822 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:18.012906 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.014589 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.015664 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.017487 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.018150 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.018269 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.023085 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.023094 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.023100 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.023140 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.025978 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.026616 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.026623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.026626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.026764 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.027162 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.027168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.027171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.027419 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.027713 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.027989 (+ 276us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.028250 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 19:36:18.033283 20654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c: Bootstrap starting.
I20260926 19:36:18.033469 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:36:18.036902 20654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:18.038038 20654 log.cc:824] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:18.041496 20654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c: No bootstrap required, opened a new log
I20260926 19:36:18.052222 20654 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.052696 20654 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:18.052827 20654 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba47a1c6672847688001aa37186ec56c, State: Initialized, Role: FOLLOWER
I20260926 19:36:18.053687 20654 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.060213 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.038181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.6.215.253:37343)
0926 19:36:18.038614 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:18.038620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:18.039716 (+ 1096us) server_negotiation.cc:407] Connection header received
0926 19:36:18.039796 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:18.039804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:18.039908 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:18.040015 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.042932 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.044079 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.046636 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.047655 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.047787 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.053004 (+ 5217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.053015 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.053022 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.053096 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.056683 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.057614 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.057620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.057623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.057778 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.058283 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.058288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.058292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.058726 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.058967 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.059619 (+ 652us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.059862 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313}
I20260926 19:36:18.060796 20668 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.061198 20668 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:36:18.063181 20654 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:18.065078 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.038372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51672 (local address 127.6.215.254:36285)
0926 19:36:18.038626 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:18.038635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:18.044794 (+ 6159us) server_negotiation.cc:407] Connection header received
0926 19:36:18.045090 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:18.045098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:18.045183 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:18.045255 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.047936 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.049015 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.051227 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.051836 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.051933 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.057023 (+ 5090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.057034 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.057042 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.057101 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.061235 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.062506 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.062514 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.062519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.062671 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.063149 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.063156 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.063162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.063524 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.063761 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.064386 (+ 625us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.064668 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 19:36:18.075314 20679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:36:18.075562 20679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:36:18.075685 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.045073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49036 (local address 127.6.215.252:45455)
0926 19:36:18.045244 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:18.045250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:18.057100 (+ 11850us) server_negotiation.cc:407] Connection header received
0926 19:36:18.057372 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:18.057382 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:18.057468 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:18.057539 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.059383 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.060492 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.063136 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.063729 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.063879 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.067504 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.067510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.067515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.067556 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.071454 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.072204 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.072208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.072210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.072444 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.073019 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.073024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.073027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.073449 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.074794 (+ 1345us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.074936 (+ 142us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.075197 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 19:36:18.072500 20587 master_runner.cc:186] Error getting master registration for 127.6.215.254:36285: OK, instance_id { permanent_uuid: "ace1946523b645539e4c501317048993" instance_seqno: 1790451376977020 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:36:18.081598 20587 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:36:18.116247 20506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 4
Sep 26 19:36:18 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451376, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
I20260926 19:36:18.131990 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.116902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:49040)
0926 19:36:18.117084 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:18.117106 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:18.117207 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:18.117562 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:18.117566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:18.117931 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:18.118093 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.118102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.119524 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.119528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.120455 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.120467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.121476 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.121481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.121737 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.122371 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:18.122389 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:18.125635 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:18.129602 (+ 3967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.129607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.129610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.130011 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.130446 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.130450 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.130452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.130593 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.131208 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:18.131211 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:18.131340 (+ 129us) client_negotiation.cc:769] Sending connection context
0926 19:36:18.131450 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 19:36:18.131718 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 19:36:18.132215 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.116957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.6.215.252:45455)
0926 19:36:18.117139 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:18.117145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:18.117167 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:36:18.117322 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:18.117328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:18.117423 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:18.117530 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.118238 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.119380 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.120607 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.121340 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.121468 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.125796 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.125802 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.125807 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.125846 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.129448 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.130147 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.130153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.130157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.130332 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.130699 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.130705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.130708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.131084 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.131234 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.131582 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.131856 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:36:18.138285 20506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: Bootstrap starting.
I20260926 19:36:18.142236 20506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:18.143536 20506 log.cc:824] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:18.146793 20506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: No bootstrap required, opened a new log
I20260926 19:36:18.157889 20506 raft_consensus.cc:348] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.158308 20506 raft_consensus.cc:374] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:18.158427 20506 raft_consensus.cc:729] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace1946523b645539e4c501317048993, State: Initialized, Role: FOLLOWER
I20260926 19:36:18.159098 20506 consensus_queue.cc:260] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.159302 20668 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:36:18.159668 20668 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.160158 20682 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.160620 20682 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:36:18.161306 20668 leader_election.cc:288] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ace1946523b645539e4c501317048993 (127.6.215.254:36285), 3b21a405716b4fbba161634d4d284e6b (127.6.215.253:37343)
I20260926 19:36:18.161954 20506 sys_catalog.cc:564] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:18.162082 20481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba47a1c6672847688001aa37186ec56c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace1946523b645539e4c501317048993" is_pre_election: true
I20260926 19:36:18.162652 20481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba47a1c6672847688001aa37186ec56c in term 0.
I20260926 19:36:18.163342 20606 leader_election.cc:302] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace1946523b645539e4c501317048993, ba47a1c6672847688001aa37186ec56c; no voters:
I20260926 19:36:18.163806 20668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:36:18.163959 20668 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:36:18.164078 20668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:18.167896 20668 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.169169 20481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba47a1c6672847688001aa37186ec56c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ace1946523b645539e4c501317048993"
I20260926 19:36:18.169389 20481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:18.172807 20481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba47a1c6672847688001aa37186ec56c in term 1.
I20260926 19:36:18.173239 20606 leader_election.cc:302] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ace1946523b645539e4c501317048993, ba47a1c6672847688001aa37186ec56c; no voters:
I20260926 19:36:18.174217 20685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:18.174675 20668 leader_election.cc:288] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [CANDIDATE]: Term 1 election: Requested vote from peers ace1946523b645539e4c501317048993 (127.6.215.254:36285), 3b21a405716b4fbba161634d4d284e6b (127.6.215.253:37343)
I20260926 19:36:18.176029 20685 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [term 1 LEADER]: Becoming Leader. State: Replica: ba47a1c6672847688001aa37186ec56c, State: Running, Role: LEADER
I20260926 19:36:18.176878 20685 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.180218 20668 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba47a1c6672847688001aa37186ec56c. Latest consensus state: current_term: 1 leader_uuid: "ba47a1c6672847688001aa37186ec56c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.180552 20668 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:18.185112 20689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:18.187474 20689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:18.192173 20580 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } attempt: 5
W20260926 19:36:18.196209 20608 tablet.cc:2369] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 19:36:18 dist-test-slave-phbh krb5kdc[20426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451377, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
I20260926 19:36:18.211858 20481 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [term 1 FOLLOWER]: Refusing update from remote peer ba47a1c6672847688001aa37186ec56c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:18.212455 20698 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:36:18.212869 20668 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:36:18.216526 20460 tablet.cc:2369] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:36:18.218600 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.192946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49046 (local address 127.6.215.252:45455)
0926 19:36:18.193471 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:18.193479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:18.193498 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:36:18.193560 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:18.193565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:18.193636 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:18.193720 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:18.194836 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.196348 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.198044 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.198694 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.198810 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.204947 (+ 6137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:18.204957 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:18.204964 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:18.205020 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:18.209055 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.210493 (+ 1438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.210536 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.210540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.210827 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:18.211692 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:18.211696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:18.211700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:18.212705 (+ 1005us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:18.213360 (+ 655us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:18.217806 (+ 4446us) server_negotiation.cc:299] Negotiation successful
0926 19:36:18.218178 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412}
I20260926 19:36:18.219601 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:18.192912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:45455 (local address 127.0.0.1:49046)
0926 19:36:18.193125 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:18.193148 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:18.193262 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:18.193898 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:18.193903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:18.194460 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:18.194681 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.194693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.196501 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.196506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.197901 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:18.197914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:18.200175 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:18.200183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:18.200445 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:18.201190 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:18.201212 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:18.204604 (+ 3392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:18.209320 (+ 4716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.209325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.209329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.210303 (+ 974us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.211159 (+ 856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:18.211163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:18.211168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:18.211489 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:18.213448 (+ 1959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:18.213453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:18.213782 (+ 329us) client_negotiation.cc:769] Sending connection context
0926 19:36:18.218865 (+ 5083us) client_negotiation.cc:240] Negotiation successful
0926 19:36:18.219222 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 19:36:18.225512 20580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b: Bootstrap starting.
I20260926 19:36:18.216466 20439 master_runner.cc:186] Error getting master registration for 127.6.215.253:37343: OK, instance_id { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" instance_seqno: 1790451377465958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:36:18.229473 20439 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:36:18.229663 20580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:18.231024 20580 log.cc:824] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:18.236949 20682 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba47a1c6672847688001aa37186ec56c. Latest consensus state: current_term: 1 leader_uuid: "ba47a1c6672847688001aa37186ec56c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.237231 20682 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:36:18.240643 20580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b: No bootstrap required, opened a new log
I20260926 19:36:18.240656 20694 mvcc.cc:205] Tried to move back new op lower bound from 7333688845140099072 to 7333688845017726976. Current Snapshot: MvccSnapshot[applied={T|T < 7333688845140099072}]
I20260926 19:36:18.241896 20685 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba47a1c6672847688001aa37186ec56c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.242210 20685 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:18.247377 20689 catalog_manager.cc:1347] Generated new cluster ID: 29cb211df38d463db0cd10c37540a25a
I20260926 19:36:18.247494 20689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:36:18.251917 20580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.252063 20682 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba47a1c6672847688001aa37186ec56c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.252305 20682 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:36:18.252333 20580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:18.252439 20580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b21a405716b4fbba161634d4d284e6b, State: Initialized, Role: FOLLOWER
I20260926 19:36:18.253100 20580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } }
I20260926 19:36:18.257990 20704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.258715 20704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:36:18.259193 20580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:18.259536 20553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:18.260942 20555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba47a1c6672847688001aa37186ec56c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b21a405716b4fbba161634d4d284e6b" is_pre_election: true
I20260926 19:36:18.262355 20554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba47a1c6672847688001aa37186ec56c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b21a405716b4fbba161634d4d284e6b"
I20260926 19:36:18.265475 20553 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [term 1 FOLLOWER]: Refusing update from remote peer ba47a1c6672847688001aa37186ec56c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:18.266292 20686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:18.279306 20689 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:36:18.282368 20689 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:36:18.286950 20686 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ba47a1c6672847688001aa37186ec56c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.287370 20686 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:18.286180 20709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ba47a1c6672847688001aa37186ec56c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.287961 20709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:36:18.287519 20704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba47a1c6672847688001aa37186ec56c. Latest consensus state: current_term: 1 leader_uuid: "ba47a1c6672847688001aa37186ec56c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ace1946523b645539e4c501317048993" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36285 } } peers { permanent_uuid: "3b21a405716b4fbba161634d4d284e6b" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 37343 } } peers { permanent_uuid: "ba47a1c6672847688001aa37186ec56c" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 45455 } } }
I20260926 19:36:18.288324 20704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:36:18.303622 20689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba47a1c6672847688001aa37186ec56c: Generated new TSK 0
I20260926 19:36:18.304811 20689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:36:18.316124 20720 catalog_manager.cc:1259] Loaded cluster ID: 29cb211df38d463db0cd10c37540a25a
I20260926 19:36:18.316246 20720 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b: loading cluster ID for follower catalog manager: success
I20260926 19:36:18.320011 20720 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b: acquiring CA information for follower catalog manager: success
I20260926 19:36:18.322733 20720 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b21a405716b4fbba161634d4d284e6b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:36:19.217458 20698 catalog_manager.cc:1259] Loaded cluster ID: 29cb211df38d463db0cd10c37540a25a
I20260926 19:36:19.217583 20698 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: loading cluster ID for follower catalog manager: success
I20260926 19:36:19.221315 20698 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: acquiring CA information for follower catalog manager: success
I20260926 19:36:19.223822 20698 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ace1946523b645539e4c501317048993: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:36:19.404258 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20439
I20260926 19:36:19.405746 20501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:19.529413 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20439
I20260926 19:36:19.548599 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20513
I20260926 19:36:19.549777 20575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:19.676188 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20513
I20260926 19:36:19.696834 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20587
I20260926 19:36:19.697950 20649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:19.806342 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20587
2026-09-26T19:36:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4614 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:36:19 dist-test-slave-phbh krb5kdc[20732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:36:19 dist-test-slave-phbh krb5kdc[20732](info): set up 2 sockets
Sep 26 19:36:19 dist-test-slave-phbh krb5kdc[20732](info): commencing operation
krb5kdc: starting...
W20260926 19:36:20.913324 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:36:21 dist-test-slave-phbh krb5kdc[20732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:36:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:36:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:36:21.147943 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:44923
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:43163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:44923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:21.478389 20745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:21.478754 20745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:21.478924 20745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:21.486093 20745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:21.486248 20745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:21.486363 20745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:21.486477 20745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:21.486567 20745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:21.496483 20745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:43163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:44923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:44923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:21.497907 20745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:21.499293 20745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:21.511834 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:21.511924 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:21.511888 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:21.512840 20745 server_base.cc:1034] running on GCE node
I20260926 19:36:21.513696 20745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:21.515133 20745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:21.516398 20745 hybrid_clock.cc:648] HybridClock initialized: now 1790451381516331 us; error 41 us; skew 500 ppm
I20260926 19:36:21.516887 20745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:21 dist-test-slave-phbh krb5kdc[20732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451381, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:21.520021 20745 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:21.521674 20745 webserver.cc:460] Webserver started at http://127.6.215.254:37861/ using document root <none> and password file <none>
I20260926 19:36:21.522300 20745 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:21.522401 20745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:21.522665 20745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:21.525652 20745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "d045a88fcded4ad5a2f4cbb6037b5953"
format_stamp: "Formatted at 2026-09-26 19:36:21 on dist-test-slave-phbh"
server_key: "496800702ec8818daa48e863b60f53ce"
server_key_iv: "1b0c4617f0b40f7a7e509d5ba9097b78"
server_key_version: "encryptionkey@0"
I20260926 19:36:21.526345 20745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "d045a88fcded4ad5a2f4cbb6037b5953"
format_stamp: "Formatted at 2026-09-26 19:36:21 on dist-test-slave-phbh"
server_key: "496800702ec8818daa48e863b60f53ce"
server_key_iv: "1b0c4617f0b40f7a7e509d5ba9097b78"
server_key_version: "encryptionkey@0"
I20260926 19:36:21.531548 20745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:36:21.534950 20760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:21.536396 20745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:36:21.536646 20745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "d045a88fcded4ad5a2f4cbb6037b5953"
format_stamp: "Formatted at 2026-09-26 19:36:21 on dist-test-slave-phbh"
server_key: "496800702ec8818daa48e863b60f53ce"
server_key_iv: "1b0c4617f0b40f7a7e509d5ba9097b78"
server_key_version: "encryptionkey@0"
I20260926 19:36:21.536927 20745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:21.566524 20745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:21.570590 20745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:21.570905 20745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:21.590003 20745 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:44923
I20260926 19:36:21.590010 20811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:44923 every 8 connection(s)
I20260926 19:36:21.591504 20745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:21.595325 20812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:21.601013 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20745
I20260926 19:36:21.601259 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:36:21.601894 7007 external_mini_cluster.cc:1428] Setting key 63422a5a04e2aba78062c2499c2579e4
I20260926 19:36:21.607043 20812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953: Bootstrap starting.
I20260926 19:36:21.610548 20812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:21.611578 20812 log.cc:824] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:21.614431 20812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953: No bootstrap required, opened a new log
Sep 26 19:36:21 dist-test-slave-phbh krb5kdc[20732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:21.623780 20812 raft_consensus.cc:348] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44923 } }
I20260926 19:36:21.624258 20812 raft_consensus.cc:374] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:21.624392 20812 raft_consensus.cc:729] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d045a88fcded4ad5a2f4cbb6037b5953, State: Initialized, Role: FOLLOWER
I20260926 19:36:21.625212 20812 consensus_queue.cc:260] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44923 } }
I20260926 19:36:21.625463 20812 raft_consensus.cc:388] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:36:21.625653 20812 raft_consensus.cc:482] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:36:21.625846 20812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:21.626324 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:21.603883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.6.215.254:44923)
0926 19:36:21.604726 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:21.604737 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:21.604830 (+ 93us) server_negotiation.cc:407] Connection header received
0926 19:36:21.607414 (+ 2584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:21.607437 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:21.607978 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:21.608442 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:21.609995 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:21.611383 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:21.612834 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:21.613511 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:21.613718 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:21.618333 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:21.618341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:21.618356 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:21.618397 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:21.621640 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:21.622611 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:21.622617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:21.622623 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:21.622797 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:21.623239 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:21.623242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:21.623244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:21.623815 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:21.624040 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:21.624623 (+ 583us) server_negotiation.cc:299] Negotiation successful
0926 19:36:21.625072 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":416,"threads_started":1}
I20260926 19:36:21.628274 20812 raft_consensus.cc:504] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44923 } }
I20260926 19:36:21.628890 20812 leader_election.cc:302] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d045a88fcded4ad5a2f4cbb6037b5953; no voters:
I20260926 19:36:21.629648 20812 leader_election.cc:288] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:36:21.629750 20817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:21.630285 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:36:21.632437 20812 sys_catalog.cc:564] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:21.632859 20817 raft_consensus.cc:686] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [term 1 LEADER]: Becoming Leader. State: Replica: d045a88fcded4ad5a2f4cbb6037b5953, State: Running, Role: LEADER
I20260926 19:36:21.633435 20817 consensus_queue.cc:237] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44923 } }
I20260926 19:36:21.636564 20819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d045a88fcded4ad5a2f4cbb6037b5953. Latest consensus state: current_term: 1 leader_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44923 } } }
I20260926 19:36:21.637028 20819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:21.637574 20818 sys_catalog.cc:455] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d045a88fcded4ad5a2f4cbb6037b5953" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44923 } } }
I20260926 19:36:21.637843 20818 sys_catalog.cc:458] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:21.638974 20830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:21.645156 20830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:21.653368 20830 catalog_manager.cc:1347] Generated new cluster ID: 89f861d2cf9d477384e24c4ba6832c62
I20260926 19:36:21.653486 20830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:36:21.667448 20830 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:36:21.668936 20830 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:36:21.676287 20830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d045a88fcded4ad5a2f4cbb6037b5953: Generated new TSK 0
I20260926 19:36:21.677204 20830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:36:21.691169 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20745
I20260926 19:36:21.692386 20807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:21.783659 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20745
2026-09-26T19:36:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1973 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:36:21 dist-test-slave-phbh krb5kdc[20840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:36:21 dist-test-slave-phbh krb5kdc[20840](info): set up 2 sockets
Sep 26 19:36:21 dist-test-slave-phbh krb5kdc[20840](info): commencing operation
krb5kdc: starting...
W20260926 19:36:22.881482 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:36:23 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:36:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:36:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:36:23.122068 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42269
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:23.466173 20853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:23.466480 20853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:23.466634 20853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:23.474090 20853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:23.474205 20853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:23.474287 20853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:23.474372 20853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:23.474433 20853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:23.484591 20853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:23.486229 20853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:23.487566 20853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:23.499771 20858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:23.499724 20859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:23.499835 20861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:23.500679 20853 server_base.cc:1034] running on GCE node
I20260926 19:36:23.501565 20853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:23.503062 20853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:23.504309 20853 hybrid_clock.cc:648] HybridClock initialized: now 1790451383504267 us; error 55 us; skew 500 ppm
I20260926 19:36:23.504669 20853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:23 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451383, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:23.513600 20853 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:23.515111 20853 webserver.cc:460] Webserver started at http://127.6.215.254:37115/ using document root <none> and password file <none>
I20260926 19:36:23.515766 20853 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:23.515864 20853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:23.516144 20853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:23.519158 20853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "2f1746b667c046d183e53e9b91b979e5"
format_stamp: "Formatted at 2026-09-26 19:36:23 on dist-test-slave-phbh"
server_key: "9aa81ae6055e7c594522b22c3301a2b7"
server_key_iv: "70c3ad0a8143a46415b1bc77859ef59b"
server_key_version: "encryptionkey@0"
I20260926 19:36:23.519850 20853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "2f1746b667c046d183e53e9b91b979e5"
format_stamp: "Formatted at 2026-09-26 19:36:23 on dist-test-slave-phbh"
server_key: "9aa81ae6055e7c594522b22c3301a2b7"
server_key_iv: "70c3ad0a8143a46415b1bc77859ef59b"
server_key_version: "encryptionkey@0"
I20260926 19:36:23.524982 20853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 19:36:23.528307 20868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:23.529716 20853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:23.529856 20853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2f1746b667c046d183e53e9b91b979e5"
format_stamp: "Formatted at 2026-09-26 19:36:23 on dist-test-slave-phbh"
server_key: "9aa81ae6055e7c594522b22c3301a2b7"
server_key_iv: "70c3ad0a8143a46415b1bc77859ef59b"
server_key_version: "encryptionkey@0"
I20260926 19:36:23.530017 20853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:23.546110 20853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:23.550200 20853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:23.550431 20853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:23.569324 20853 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42269
I20260926 19:36:23.569365 20919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42269 every 8 connection(s)
I20260926 19:36:23.570735 20853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:23.574445 20920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:23.575670 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20853
I20260926 19:36:23.575909 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:36:23.576828 7007 external_mini_cluster.cc:1428] Setting key b08230cc2f7456736f089806192b889d
I20260926 19:36:23.587836 20920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Bootstrap starting.
Sep 26 19:36:23 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:23.590957 20920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:23.591988 20920 log.cc:824] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:23.596203 20920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: No bootstrap required, opened a new log
I20260926 19:36:23.599581 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:23.579060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.6.215.254:42269)
0926 19:36:23.579883 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:23.579898 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:23.579937 (+ 39us) server_negotiation.cc:407] Connection header received
0926 19:36:23.581630 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:23.581652 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:23.582060 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:23.582463 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:23.583916 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:23.585318 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:23.586691 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:23.587360 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:23.587558 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:23.591648 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:23.591656 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:23.591671 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:23.591710 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:23.595111 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:23.595865 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:23.595872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:23.595878 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:23.596047 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:23.596499 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:23.596506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:23.596509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:23.597202 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:23.597526 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:23.597984 (+ 458us) server_negotiation.cc:299] Negotiation successful
0926 19:36:23.598465 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":398,"threads_started":1}
I20260926 19:36:23.605995 20920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:23.606413 20920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:23.606534 20920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1746b667c046d183e53e9b91b979e5, State: Initialized, Role: FOLLOWER
I20260926 19:36:23.607208 20920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:23.607460 20920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:36:23.607602 20920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:36:23.607784 20920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:23.610057 20920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:23.610590 20920 leader_election.cc:302] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f1746b667c046d183e53e9b91b979e5; no voters:
I20260926 19:36:23.611296 20920 leader_election.cc:288] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:36:23.611423 20925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:23.612160 20925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f1746b667c046d183e53e9b91b979e5, State: Running, Role: LEADER
I20260926 19:36:23.612699 20925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:23.613708 20920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:23.625523 20926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f1746b667c046d183e53e9b91b979e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } } }
I20260926 19:36:23.626025 20926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:23.627425 20927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f1746b667c046d183e53e9b91b979e5. Latest consensus state: current_term: 1 leader_uuid: "2f1746b667c046d183e53e9b91b979e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } } }
I20260926 19:36:23.627792 20927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:23.628827 20931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:23.634909 20931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:23.644150 20931 catalog_manager.cc:1347] Generated new cluster ID: 59f997454537456d839ef083edcfa486
I20260926 19:36:23.644281 20931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:36:23.654676 20931 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:36:23.655931 20931 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:36:23.661509 20931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Generated new TSK 0
I20260926 19:36:23.662596 20931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:36:23.723811 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42269
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:24.060628 20946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:24.061049 20946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:24.061188 20946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:24.069288 20946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:24.069404 20946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:24.069525 20946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:36:24.079525 20946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:24.080878 20946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:24.082213 20946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:24.095266 20951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:24.095299 20952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:24.095582 20954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:24.096642 20946 server_base.cc:1034] running on GCE node
I20260926 19:36:24.097400 20946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:24.098515 20946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:24.099738 20946 hybrid_clock.cc:648] HybridClock initialized: now 1790451384099668 us; error 71 us; skew 500 ppm
I20260926 19:36:24.100113 20946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:24 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451384, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:24.103328 20946 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:36:24.104954 20946 webserver.cc:460] Webserver started at http://127.6.215.193:35479/ using document root <none> and password file <none>
I20260926 19:36:24.105674 20946 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:24.105809 20946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:24.106122 20946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:24.109128 20946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "60bf19187c744aceb7bac29400c76aee"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "998a8485c0706d4a220d567a01c9043e"
server_key_iv: "9dcffec87734ca201a41d12b579558d0"
server_key_version: "encryptionkey@0"
I20260926 19:36:24.109891 20946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "60bf19187c744aceb7bac29400c76aee"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "998a8485c0706d4a220d567a01c9043e"
server_key_iv: "9dcffec87734ca201a41d12b579558d0"
server_key_version: "encryptionkey@0"
I20260926 19:36:24.115221 20946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:36:24.118793 20961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:24.120167 20946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:36:24.120337 20946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "60bf19187c744aceb7bac29400c76aee"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "998a8485c0706d4a220d567a01c9043e"
server_key_iv: "9dcffec87734ca201a41d12b579558d0"
server_key_version: "encryptionkey@0"
I20260926 19:36:24.120558 20946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:24.144497 20946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:24.148554 20946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:24.148908 20946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:24.149893 20946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:24.151194 20946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:24.151265 20946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:24.151430 20946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:24.151491 20946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:24.184715 20946 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:35305
I20260926 19:36:24.184780 21073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:35305 every 8 connection(s)
I20260926 19:36:24.186025 20946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:36:24.188115 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 20946
I20260926 19:36:24.188334 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:36:24.189188 7007 external_mini_cluster.cc:1428] Setting key b3a0aeafea5a476008277c502be32e14
Sep 26 19:36:24 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451384, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:24.208056 21076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:24.188708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:33147 (local address 127.6.215.254:42269)
0926 19:36:24.189223 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:24.189227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:24.190259 (+ 1032us) server_negotiation.cc:407] Connection header received
0926 19:36:24.192417 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:24.192423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:24.192530 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:24.192651 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:24.194160 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.194887 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.196449 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.196924 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.197054 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:24.201030 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:24.201039 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:24.201047 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:24.201092 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:24.204691 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:24.205461 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:24.205469 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:24.205476 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:24.205645 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:24.206048 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:24.206056 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:24.206062 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:24.206438 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:24.206613 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:24.207264 (+ 651us) server_negotiation.cc:299] Negotiation successful
0926 19:36:24.207526 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":293,"threads_started":1}
I20260926 19:36:24.208657 21077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:24.189241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:33147)
0926 19:36:24.190090 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:24.190145 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:24.192150 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:24.192911 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:24.192920 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:24.193425 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:24.193987 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:24.194003 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.195040 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.195046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:24.196285 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:24.196296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.197155 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.197164 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:24.197399 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:24.198097 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:24.198120 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:24.200756 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:24.204870 (+ 4114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:24.204877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:24.204892 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:24.205323 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:24.205753 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:24.205757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:24.205759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:24.205931 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:24.206561 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:24.206576 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:24.206876 (+ 300us) client_negotiation.cc:769] Sending connection context
0926 19:36:24.207093 (+ 217us) client_negotiation.cc:240] Negotiation successful
0926 19:36:24.207470 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":317,"threads_started":1}
I20260926 19:36:24.210371 21074 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42269
I20260926 19:36:24.210691 21074 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:24.211373 21074 heartbeater.cc:511] Master 127.6.215.254:42269 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
I20260926 19:36:24.213485 20885 ts_manager.cc:194] Registered new tserver with Master: 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305)
I20260926 19:36:24.215242 20885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:33147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:36:24.243860 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42269
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:24.577170 21080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:24.577601 21080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:24.577782 21080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:24.586328 21080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:24.586447 21080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:24.586565 21080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:36:24.596345 21080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:24.597704 21080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:24.598935 21080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:24.612021 21085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:24.612113 21088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:24.612049 21086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:24.612617 21080 server_base.cc:1034] running on GCE node
I20260926 19:36:24.613318 21080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:24.614347 21080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:24.615584 21080 hybrid_clock.cc:648] HybridClock initialized: now 1790451384615542 us; error 58 us; skew 500 ppm
I20260926 19:36:24.615924 21080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:24 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451384, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:24.619181 21080 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:36:24.620728 21080 webserver.cc:460] Webserver started at http://127.6.215.194:40267/ using document root <none> and password file <none>
I20260926 19:36:24.621486 21080 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:24.621634 21080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:24.621938 21080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:24.625135 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "26c348a5f9f04f56890369df08809928"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "565ab0ca36dcf283969de700c4590478"
server_key_iv: "3fe7cc226a2012b1fc657fed83e4640e"
server_key_version: "encryptionkey@0"
I20260926 19:36:24.626212 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "26c348a5f9f04f56890369df08809928"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "565ab0ca36dcf283969de700c4590478"
server_key_iv: "3fe7cc226a2012b1fc657fed83e4640e"
server_key_version: "encryptionkey@0"
I20260926 19:36:24.632146 21080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:36:24.635752 21095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:24.637223 21080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 19:36:24.637398 21080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "26c348a5f9f04f56890369df08809928"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "565ab0ca36dcf283969de700c4590478"
server_key_iv: "3fe7cc226a2012b1fc657fed83e4640e"
server_key_version: "encryptionkey@0"
I20260926 19:36:24.637602 21080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:24.667748 21080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:24.671876 21080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:24.672200 21080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:24.673281 21080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:24.674638 21080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:24.674746 21080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:24.674896 21080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:24.674978 21080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:24.707996 21080 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:34299
I20260926 19:36:24.708038 21207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:34299 every 8 connection(s)
I20260926 19:36:24.709429 21080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:36:24.716876 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21080
I20260926 19:36:24.717108 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:36:24.717561 7007 external_mini_cluster.cc:1428] Setting key 7c709ae01cf6d8a9bcb7cd2aee732e52
Sep 26 19:36:24 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451384, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:24.731319 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:24.711879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:42225 (local address 127.6.215.254:42269)
0926 19:36:24.712356 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:24.712361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:24.713213 (+ 852us) server_negotiation.cc:407] Connection header received
0926 19:36:24.715131 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:24.715137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:24.715222 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:24.715343 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:24.716990 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.717839 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.719740 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.720154 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.720267 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:24.724519 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:24.724524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:24.724527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:24.724558 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:24.727851 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:24.728883 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:24.728887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:24.728889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:24.729050 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:24.729575 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:24.729578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:24.729580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:24.729877 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:24.730047 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:24.730877 (+ 830us) server_negotiation.cc:299] Negotiation successful
0926 19:36:24.731044 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":265,"threads_started":1}
I20260926 19:36:24.732925 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:24.712351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:42225)
0926 19:36:24.713049 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:24.713092 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:24.714838 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:24.715577 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:24.715585 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:24.716049 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:24.716757 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:24.716815 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.717990 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.717994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:24.719599 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:24.719612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:24.720400 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:24.720409 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:24.720743 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:24.721504 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:24.721537 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:24.724200 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:24.727994 (+ 3794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:24.728005 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:24.728025 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:24.728702 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:24.729173 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:24.729180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:24.729187 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:24.729460 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:24.730030 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:24.730040 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:24.730502 (+ 462us) client_negotiation.cc:769] Sending connection context
0926 19:36:24.730800 (+ 298us) client_negotiation.cc:240] Negotiation successful
0926 19:36:24.731401 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":269,"threads_started":1}
I20260926 19:36:24.734556 21208 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42269
I20260926 19:36:24.734890 21208 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:24.735632 21208 heartbeater.cc:511] Master 127.6.215.254:42269 requested a full tablet report, sending...
I20260926 19:36:24.737308 20885 ts_manager.cc:194] Registered new tserver with Master: 26c348a5f9f04f56890369df08809928 (127.6.215.194:34299)
I20260926 19:36:24.738178 20885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:42225
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:36:24.772023 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42269
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:25.098511 21214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:25.098896 21214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:25.099081 21214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:25.106792 21214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:25.106945 21214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:25.107142 21214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:36:25.117328 21214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:25.118666 21214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:25.120024 21214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:25.133740 21220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:25.133986 21219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:25.133961 21222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:25.134745 21214 server_base.cc:1034] running on GCE node
I20260926 19:36:25.135327 21214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:25.136263 21214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:25.137518 21214 hybrid_clock.cc:648] HybridClock initialized: now 1790451385137450 us; error 60 us; skew 500 ppm
I20260926 19:36:25.137863 21214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:25 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451385, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:25.141091 21214 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:36:25.142488 21214 webserver.cc:460] Webserver started at http://127.6.215.195:42665/ using document root <none> and password file <none>
I20260926 19:36:25.143146 21214 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:25.143242 21214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:25.143534 21214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:25.146598 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "659ef73a1d214766836c172c8f5992c3"
format_stamp: "Formatted at 2026-09-26 19:36:25 on dist-test-slave-phbh"
server_key: "1b7fc7a247e7e1dd5bfd16bef09030df"
server_key_iv: "f24e3753fd29654d1c00ef03036ad833"
server_key_version: "encryptionkey@0"
I20260926 19:36:25.147297 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "659ef73a1d214766836c172c8f5992c3"
format_stamp: "Formatted at 2026-09-26 19:36:25 on dist-test-slave-phbh"
server_key: "1b7fc7a247e7e1dd5bfd16bef09030df"
server_key_iv: "f24e3753fd29654d1c00ef03036ad833"
server_key_version: "encryptionkey@0"
I20260926 19:36:25.152359 21214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:36:25.155836 21229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:25.157178 21214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 19:36:25.157303 21214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "659ef73a1d214766836c172c8f5992c3"
format_stamp: "Formatted at 2026-09-26 19:36:25 on dist-test-slave-phbh"
server_key: "1b7fc7a247e7e1dd5bfd16bef09030df"
server_key_iv: "f24e3753fd29654d1c00ef03036ad833"
server_key_version: "encryptionkey@0"
I20260926 19:36:25.157461 21214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:25.172060 21214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:25.175966 21214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:25.176203 21214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:25.177227 21214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:25.178532 21214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:25.178615 21214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:25.178702 21214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:25.178745 21214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:25.211066 21214 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:44387
I20260926 19:36:25.211086 21341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:44387 every 8 connection(s)
I20260926 19:36:25.212355 21214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:36:25.215045 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21214
I20260926 19:36:25.215279 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:36:25.215723 7007 external_mini_cluster.cc:1428] Setting key 3155ed886dcdcbf771d73c94daba1af5
I20260926 19:36:25.218645 21074 heartbeater.cc:503] Master 127.6.215.254:42269 was elected leader, sending a full tablet report...
Sep 26 19:36:25 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451385, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:25.232970 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.214863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:43035 (local address 127.6.215.254:42269)
0926 19:36:25.215091 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.215098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.216333 (+ 1235us) server_negotiation.cc:407] Connection header received
0926 19:36:25.218546 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.218551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.218644 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.218749 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:25.220385 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.221209 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.222831 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.223247 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.223337 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.226791 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:25.226796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:25.226799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:25.226844 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:25.229491 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.230380 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.230386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.230395 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.230547 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.231047 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.231053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.231057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.231458 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:25.231644 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.232377 (+ 733us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.232608 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 19:36:25.234109 21344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.215285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:43035)
0926 19:36:25.216144 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:25.216198 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:25.218222 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:25.219150 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:25.219159 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:25.219673 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:25.220234 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.220252 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.221370 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.221376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.222710 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.222722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.223437 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.223441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.223681 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.224319 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:25.224348 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:25.226582 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:25.229641 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:25.229648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:25.229661 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:25.230249 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:25.230681 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:25.230686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:25.230689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:25.230930 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:25.231606 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:25.231616 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:25.232003 (+ 387us) client_negotiation.cc:769] Sending connection context
0926 19:36:25.232280 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 19:36:25.232864 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":340,"threads_started":1}
I20260926 19:36:25.235605 21342 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42269
I20260926 19:36:25.235911 21342 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:25.236655 21342 heartbeater.cc:511] Master 127.6.215.254:42269 requested a full tablet report, sending...
I20260926 19:36:25.238161 20885 ts_manager.cc:194] Registered new tserver with Master: 659ef73a1d214766836c172c8f5992c3 (127.6.215.195:44387)
I20260926 19:36:25.238971 20885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:43035
I20260926 19:36:25.243602 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:36:25.260882 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.248114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.6.215.254:42269)
0926 19:36:25.248287 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.248292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.248836 (+ 544us) server_negotiation.cc:407] Connection header received
0926 19:36:25.248991 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.248994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.249084 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.249173 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:25.250571 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.251293 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.252620 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.253099 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.253193 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.255612 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:25.255617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:25.255621 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:25.255654 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:25.258282 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.258991 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.258995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.258997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.259133 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.259481 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.259485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.259487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.259814 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:25.260012 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.260275 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.260510 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:36:25.264935 20885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:36:25.271854 20885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:36:25.294189 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.284198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:44387 (local address 127.0.0.1:51738)
0926 19:36:25.285304 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:25.285354 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:25.285585 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:25.286960 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:25.286969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:25.287004 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:25.287403 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.287415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.289642 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.289647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.291706 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.291718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.293551 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.293556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.293682 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.293720 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 19:36:25.293800 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 19:36:25.293924 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":578,"threads_started":1}
I20260926 19:36:25.295396 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.284458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.6.215.195:44387)
0926 19:36:25.286283 (+ 1825us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.286293 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.286329 (+ 36us) server_negotiation.cc:407] Connection header received
0926 19:36:25.286435 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.286443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.286710 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.287598 (+ 888us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:25.287890 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.289492 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.291950 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.293335 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.294641 (+ 1306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.294830 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.294889 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.295085 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1653,"thread_start_us":1498,"threads_started":1}
I20260926 19:36:25.295529 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.284853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:43398)
0926 19:36:25.286527 (+ 1674us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:25.286544 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:25.286656 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:25.287805 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:25.287813 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:25.287838 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:25.288323 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.288337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.290457 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.290465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.292684 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.292701 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.294653 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.294660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.294833 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.294867 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 19:36:25.294940 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 19:36:25.295092 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1566,"mutex_wait_us":1,"thread_start_us":1434,"threads_started":1}
I20260926 19:36:25.295893 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.285579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:34299 (local address 127.0.0.1:39692)
0926 19:36:25.288015 (+ 2436us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:25.288030 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:25.288187 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:25.288843 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:25.288847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:25.288870 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:25.289195 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.289205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.291059 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.291073 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.293234 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.293247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.295092 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.295098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.295274 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.295304 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 19:36:25.295403 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 19:36:25.295556 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2331,"thread_start_us":2167,"threads_started":1}
I20260926 19:36:25.296128 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.285496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.6.215.193:35305)
0926 19:36:25.287205 (+ 1709us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.287213 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.287238 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:36:25.287341 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.287349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.287587 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.288408 (+ 821us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:25.288633 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.290317 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.292915 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.294388 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.295316 (+ 928us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.295537 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.295617 (+ 80us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.295799 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1544,"thread_start_us":1407,"threads_started":1}
I20260926 19:36:25.296504 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.286022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.6.215.194:34299)
0926 19:36:25.287548 (+ 1526us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.287560 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.288144 (+ 584us) server_negotiation.cc:407] Connection header received
0926 19:36:25.288376 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.288383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.288609 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.288856 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:25.289375 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.290902 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.293427 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.294864 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.295008 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.295195 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.295993 (+ 798us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.296179 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1345,"thread_start_us":1245,"threads_started":1}
I20260926 19:36:25.299546 21277 tablet_service.cc:1467] Processing CreateTablet for tablet 35dd5cead449425191ded32e8f6854ed (DEFAULT_TABLE table=test-table [id=6a63604c25c641818ad23067e27a4c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:25.299860 21009 tablet_service.cc:1467] Processing CreateTablet for tablet 35dd5cead449425191ded32e8f6854ed (DEFAULT_TABLE table=test-table [id=6a63604c25c641818ad23067e27a4c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:25.299860 21143 tablet_service.cc:1467] Processing CreateTablet for tablet 35dd5cead449425191ded32e8f6854ed (DEFAULT_TABLE table=test-table [id=6a63604c25c641818ad23067e27a4c14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:25.300935 21277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35dd5cead449425191ded32e8f6854ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:25.301218 21143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35dd5cead449425191ded32e8f6854ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:25.301220 21009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35dd5cead449425191ded32e8f6854ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:25.312457 21360 tablet_bootstrap.cc:492] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Bootstrap starting.
I20260926 19:36:25.315671 21362 tablet_bootstrap.cc:492] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Bootstrap starting.
I20260926 19:36:25.316493 21361 tablet_bootstrap.cc:492] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Bootstrap starting.
I20260926 19:36:25.317042 21360 tablet_bootstrap.cc:654] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:25.318440 21360 log.cc:824] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:25.319927 21362 tablet_bootstrap.cc:654] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:25.320004 21361 tablet_bootstrap.cc:654] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:25.321074 21362 log.cc:824] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:25.321267 21361 log.cc:824] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:25.321862 21360 tablet_bootstrap.cc:492] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: No bootstrap required, opened a new log
I20260926 19:36:25.322216 21360 ts_tablet_manager.cc:1397] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 19:36:25.324146 21361 tablet_bootstrap.cc:492] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: No bootstrap required, opened a new log
I20260926 19:36:25.324268 21362 tablet_bootstrap.cc:492] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: No bootstrap required, opened a new log
I20260926 19:36:25.324476 21361 ts_tablet_manager.cc:1397] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 19:36:25.324529 21362 ts_tablet_manager.cc:1397] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260926 19:36:25.332477 21361 raft_consensus.cc:348] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.332964 21361 raft_consensus.cc:374] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:25.333137 21361 raft_consensus.cc:729] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c348a5f9f04f56890369df08809928, State: Initialized, Role: FOLLOWER
I20260926 19:36:25.333178 21360 raft_consensus.cc:348] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.333657 21360 raft_consensus.cc:374] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:25.333755 21360 raft_consensus.cc:729] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 659ef73a1d214766836c172c8f5992c3, State: Initialized, Role: FOLLOWER
I20260926 19:36:25.333902 21361 consensus_queue.cc:260] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.334337 21360 consensus_queue.cc:260] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.335346 21208 heartbeater.cc:503] Master 127.6.215.254:42269 was elected leader, sending a full tablet report...
I20260926 19:36:25.335111 21362 raft_consensus.cc:348] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.335517 21362 raft_consensus.cc:374] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:25.335704 21362 raft_consensus.cc:729] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60bf19187c744aceb7bac29400c76aee, State: Initialized, Role: FOLLOWER
I20260926 19:36:25.336135 21361 ts_tablet_manager.cc:1428] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260926 19:36:25.336458 21362 consensus_queue.cc:260] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.338876 21362 ts_tablet_manager.cc:1428] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260926 19:36:25.341595 21342 heartbeater.cc:503] Master 127.6.215.254:42269 was elected leader, sending a full tablet report...
I20260926 19:36:25.342378 21360 ts_tablet_manager.cc:1428] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Time spent starting tablet: real 0.020s user 0.014s sys 0.003s
I20260926 19:36:25.365305 21366 raft_consensus.cc:482] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:36:25.365670 21366 raft_consensus.cc:504] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.367368 21366 leader_election.cc:288] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 659ef73a1d214766836c172c8f5992c3 (127.6.215.195:44387), 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305)
I20260926 19:36:25.374281 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.367803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:56009 (local address 127.6.215.195:44387)
0926 19:36:25.368012 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.368018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.369024 (+ 1006us) server_negotiation.cc:407] Connection header received
0926 19:36:25.369225 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.369234 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.369293 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.369391 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:25.370098 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.370821 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.372528 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.373328 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.373523 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.373611 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.373898 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.374019 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 19:36:25.374346 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.367831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:57251 (local address 127.6.215.193:35305)
0926 19:36:25.368036 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.368043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.368963 (+ 920us) server_negotiation.cc:407] Connection header received
0926 19:36:25.369193 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.369199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.369268 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.369367 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:25.369988 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.370697 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.372548 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.373414 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.373610 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.373687 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.373960 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.374087 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:36:25.374464 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.368013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:44387 (local address 127.6.215.194:56009)
0926 19:36:25.368918 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:25.368932 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:25.369090 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:25.369405 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:25.369409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:25.369431 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:25.369946 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.369955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.370961 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.370966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.372378 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.372389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.373554 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.373560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.373735 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.373759 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 19:36:25.373864 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 19:36:25.373996 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":5,"thread_start_us":448,"threads_started":1}
I20260926 19:36:25.374760 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.367963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.6.215.194:57251)
0926 19:36:25.368852 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:25.368869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:25.369037 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:25.369386 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:25.369390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:25.369414 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:25.369842 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.369851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.370834 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.370840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.372389 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:25.372400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.373642 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.373648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:25.373803 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.373825 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 19:36:25.373933 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 19:36:25.374062 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":49,"thread_start_us":462,"threads_started":1}
I20260926 19:36:25.375690 21297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dd5cead449425191ded32e8f6854ed" candidate_uuid: "26c348a5f9f04f56890369df08809928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "659ef73a1d214766836c172c8f5992c3" is_pre_election: true
I20260926 19:36:25.375712 21029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dd5cead449425191ded32e8f6854ed" candidate_uuid: "26c348a5f9f04f56890369df08809928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60bf19187c744aceb7bac29400c76aee" is_pre_election: true
I20260926 19:36:25.376222 21297 raft_consensus.cc:2454] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c348a5f9f04f56890369df08809928 in term 0.
I20260926 19:36:25.376231 21029 raft_consensus.cc:2454] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c348a5f9f04f56890369df08809928 in term 0.
I20260926 19:36:25.376971 21099 leader_election.cc:302] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c348a5f9f04f56890369df08809928, 60bf19187c744aceb7bac29400c76aee; no voters:
I20260926 19:36:25.377463 21366 raft_consensus.cc:2781] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:36:25.377596 21366 raft_consensus.cc:482] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:36:25.377673 21366 raft_consensus.cc:3037] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:25.379839 21366 raft_consensus.cc:504] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.380390 21366 leader_election.cc:288] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [CANDIDATE]: Term 1 election: Requested vote from peers 659ef73a1d214766836c172c8f5992c3 (127.6.215.195:44387), 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305)
I20260926 19:36:25.380991 21297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dd5cead449425191ded32e8f6854ed" candidate_uuid: "26c348a5f9f04f56890369df08809928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "659ef73a1d214766836c172c8f5992c3"
I20260926 19:36:25.381208 21297 raft_consensus.cc:3037] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:25.381368 21029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dd5cead449425191ded32e8f6854ed" candidate_uuid: "26c348a5f9f04f56890369df08809928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60bf19187c744aceb7bac29400c76aee"
I20260926 19:36:25.381597 21029 raft_consensus.cc:3037] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:25.384639 21297 raft_consensus.cc:2454] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c348a5f9f04f56890369df08809928 in term 1.
I20260926 19:36:25.384824 21029 raft_consensus.cc:2454] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c348a5f9f04f56890369df08809928 in term 1.
I20260926 19:36:25.385244 21097 leader_election.cc:302] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c348a5f9f04f56890369df08809928, 659ef73a1d214766836c172c8f5992c3; no voters:
I20260926 19:36:25.385624 21366 raft_consensus.cc:2781] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:25.386181 21366 raft_consensus.cc:686] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 1 LEADER]: Becoming Leader. State: Replica: 26c348a5f9f04f56890369df08809928, State: Running, Role: LEADER
I20260926 19:36:25.386668 21366 consensus_queue.cc:237] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } }
I20260926 19:36:25.391408 20885 catalog_manager.cc:5495] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26c348a5f9f04f56890369df08809928 (127.6.215.194). New cstate: current_term: 1 leader_uuid: "26c348a5f9f04f56890369df08809928" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } health_report { overall_health: UNKNOWN } } }
I20260926 19:36:25.423797 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.418043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.6.215.194:34299)
0926 19:36:25.418278 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.418285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.418333 (+ 48us) server_negotiation.cc:407] Connection header received
0926 19:36:25.418617 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.418622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.418701 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.418862 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:36:25.419245 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.419977 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.421490 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.421960 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.422107 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.422628 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:36:25.423162 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:36:25.423382 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.423439 (+ 57us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.423525 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 19:36:25.432305 21297 raft_consensus.cc:1261] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 1 FOLLOWER]: Refusing update from remote peer 26c348a5f9f04f56890369df08809928: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:25.432300 21029 raft_consensus.cc:1261] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Refusing update from remote peer 26c348a5f9f04f56890369df08809928: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:25.433291 21371 consensus_queue.cc:1035] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:25.433697 21366 consensus_queue.cc:1035] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [LEADER]: Connected to new peer: Peer: permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 19:36:25.438295 21075 tablet.cc:2369] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:36:25.463384 20884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:25.463711 20884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:25.467725 20884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Sending DeleteTablet for 3 replicas of tablet 35dd5cead449425191ded32e8f6854ed
I20260926 19:36:25.469004 21277 tablet_service.cc:1514] Processing DeleteTablet for tablet 35dd5cead449425191ded32e8f6854ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:25 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:51738
I20260926 19:36:25.469081 21009 tablet_service.cc:1514] Processing DeleteTablet for tablet 35dd5cead449425191ded32e8f6854ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:25 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43398
I20260926 19:36:25.469920 21384 tablet_replica.cc:330] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: stopping tablet replica
I20260926 19:36:25.470005 21143 tablet_service.cc:1514] Processing DeleteTablet for tablet 35dd5cead449425191ded32e8f6854ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:25 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:39692
I20260926 19:36:25.470139 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20946
I20260926 19:36:25.470383 21384 raft_consensus.cc:2229] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:25.471020 21384 raft_consensus.cc:2258] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:25.473676 21384 ts_tablet_manager.cc:1905] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:25.480823 21385 tablet_replica.cc:330] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: stopping tablet replica
I20260926 19:36:25.481339 21385 raft_consensus.cc:2229] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:25.482034 21386 tablet_replica.cc:330] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: stopping tablet replica
I20260926 19:36:25.482616 21386 raft_consensus.cc:2229] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:36:25.482784 21384 ts_tablet_manager.cc:1918] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:25.483014 21384 log.cc:1196] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/35dd5cead449425191ded32e8f6854ed
I20260926 19:36:25.483456 21386 raft_consensus.cc:2258] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:25.483673 21384 ts_tablet_manager.cc:1939] T 35dd5cead449425191ded32e8f6854ed P 659ef73a1d214766836c172c8f5992c3: Deleting consensus metadata
I20260926 19:36:25.484102 21385 raft_consensus.cc:2258] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:25.485788 20870 catalog_manager.cc:4841] TS 659ef73a1d214766836c172c8f5992c3 (127.6.215.195:44387): tablet 35dd5cead449425191ded32e8f6854ed (table test-table [id=6a63604c25c641818ad23067e27a4c14]) successfully deleted
I20260926 19:36:25.486696 21385 ts_tablet_manager.cc:1905] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 19:36:25 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:36:25.492174 21385 ts_tablet_manager.cc:1918] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:25.492447 21385 log.cc:1196] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/35dd5cead449425191ded32e8f6854ed
I20260926 19:36:25.492594 21386 ts_tablet_manager.cc:1905] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:25.493160 21385 ts_tablet_manager.cc:1939] T 35dd5cead449425191ded32e8f6854ed P 60bf19187c744aceb7bac29400c76aee: Deleting consensus metadata
I20260926 19:36:25.495231 20872 catalog_manager.cc:4841] TS 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305): tablet 35dd5cead449425191ded32e8f6854ed (table test-table [id=6a63604c25c641818ad23067e27a4c14]) successfully deleted
I20260926 19:36:25.500520 21357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.471409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.6.215.193:35305)
0926 19:36:25.471603 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.471609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.479516 (+ 7907us) server_negotiation.cc:407] Connection header received
0926 19:36:25.479723 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.479728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.479820 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.479943 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:25.482609 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.483808 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.486237 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.486905 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.487018 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.493685 (+ 6667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:25.493700 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:25.493716 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:25.493766 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:25.497350 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.498260 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.498270 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.498276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.498458 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.498897 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.498903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.498906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.499285 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:25.499622 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.499931 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.500154 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:36:25.500737 21386 ts_tablet_manager.cc:1918] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:25.500991 21386 log.cc:1196] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/35dd5cead449425191ded32e8f6854ed
I20260926 19:36:25.501302 21069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:25.501585 21386 ts_tablet_manager.cc:1939] T 35dd5cead449425191ded32e8f6854ed P 26c348a5f9f04f56890369df08809928: Deleting consensus metadata
I20260926 19:36:25.503700 20870 catalog_manager.cc:4841] TS 26c348a5f9f04f56890369df08809928 (127.6.215.194:34299): tablet 35dd5cead449425191ded32e8f6854ed (table test-table [id=6a63604c25c641818ad23067e27a4c14]) successfully deleted
I20260926 19:36:25.603075 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20946
I20260926 19:36:25.622036 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 21080
Sep 26 19:36:25 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:36:25.639276 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.622852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.6.215.194:34299)
0926 19:36:25.623024 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.623030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.623090 (+ 60us) server_negotiation.cc:407] Connection header received
0926 19:36:25.623295 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.623305 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.623386 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.623498 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:25.624813 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.625617 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.626974 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.627412 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.627547 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.632856 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:25.632871 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:25.632887 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:25.632962 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:25.636119 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.636983 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.636991 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.636993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.637139 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.637529 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.637536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.637539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.637910 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:25.638273 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.638646 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.638885 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:36:25.640034 21203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:25.755250 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 21080
I20260926 19:36:25.771059 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 21214
Sep 26 19:36:25 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:36:25.785502 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:25.771743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.6.215.195:44387)
0926 19:36:25.771909 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:25.771913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:25.771929 (+ 16us) server_negotiation.cc:407] Connection header received
0926 19:36:25.772062 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:25.772066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:25.772129 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:25.772211 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:25.773549 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.774287 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.775495 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:25.775881 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:25.775970 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:25.780406 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:25.780421 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:25.780437 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:25.780478 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:25.783116 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.783713 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.783720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.783722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.783846 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:25.784160 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:25.784163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:25.784165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:25.784496 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:25.784708 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:25.785097 (+ 389us) server_negotiation.cc:299] Negotiation successful
0926 19:36:25.785257 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 19:36:25.786190 21337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:25.874980 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 21214
I20260926 19:36:25.891312 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 20853
I20260926 19:36:25.892483 20915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:26.002766 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 20853
I20260926 19:36:26.024155 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42269
--webserver_interface=127.6.215.254
--webserver_port=37115
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:26.357236 21392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:26.357514 21392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:26.357681 21392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:26.365094 21392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:26.365237 21392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:26.365327 21392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:26.365453 21392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:26.365535 21392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:26.375473 21392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=37115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:26.377298 21392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:26.378737 21392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:26.401425 21398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:26.401508 21401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:26.401530 21399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:26.406494 21392 server_base.cc:1034] running on GCE node
I20260926 19:36:26.407500 21392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:26.409216 21392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:26.410461 21392 hybrid_clock.cc:648] HybridClock initialized: now 1790451386410415 us; error 55 us; skew 500 ppm
I20260926 19:36:26.410880 21392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:26 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451386, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:26.414459 21392 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:26.416373 21392 webserver.cc:460] Webserver started at http://127.6.215.254:37115/ using document root <none> and password file <none>
I20260926 19:36:26.417341 21392 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:26.417547 21392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:26.424021 21392 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 19:36:26.427208 21408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:26.428622 21392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:36:26.428871 21392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2f1746b667c046d183e53e9b91b979e5"
format_stamp: "Formatted at 2026-09-26 19:36:23 on dist-test-slave-phbh"
server_key: "9aa81ae6055e7c594522b22c3301a2b7"
server_key_iv: "70c3ad0a8143a46415b1bc77859ef59b"
server_key_version: "encryptionkey@0"
I20260926 19:36:26.429993 21392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:26.448941 21392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:26.453167 21392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:26.453469 21392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:26.473395 21392 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42269
I20260926 19:36:26.473623 21459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42269 every 8 connection(s)
I20260926 19:36:26.475312 21392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:26.477674 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21392
I20260926 19:36:26.478533 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:35305
--local_ip_for_outbound_sockets=127.6.215.193
--tserver_master_addrs=127.6.215.254:42269
--webserver_port=35479
--webserver_interface=127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:36:26.483434 21460 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:36:26.488039 21460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Bootstrap starting.
I20260926 19:36:26.509538 21460 log.cc:824] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:26.522245 21460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:36:26.522894 21460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Bootstrap complete.
I20260926 19:36:26.532275 21460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:26.533254 21460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1746b667c046d183e53e9b91b979e5, State: Initialized, Role: FOLLOWER
I20260926 19:36:26.533998 21460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:26.534260 21460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:36:26.534427 21460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:36:26.534616 21460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:36:26.539009 21460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:26.539674 21460 leader_election.cc:302] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f1746b667c046d183e53e9b91b979e5; no voters:
I20260926 19:36:26.540417 21460 leader_election.cc:288] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 19:36:26.540562 21464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:36:26.541766 21464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 LEADER]: Becoming Leader. State: Replica: 2f1746b667c046d183e53e9b91b979e5, State: Running, Role: LEADER
I20260926 19:36:26.542413 21464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:26.542928 21460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:26.544833 21465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2f1746b667c046d183e53e9b91b979e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } } }
I20260926 19:36:26.545181 21465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:26.545783 21466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f1746b667c046d183e53e9b91b979e5. Latest consensus state: current_term: 2 leader_uuid: "2f1746b667c046d183e53e9b91b979e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } } }
I20260926 19:36:26.546075 21466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:26.557394 21469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:26.566116 21469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:26.567695 21469 catalog_manager.cc:1259] Loaded cluster ID: 59f997454537456d839ef083edcfa486
I20260926 19:36:26.567863 21469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:36:26.572430 21469 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 19:36:26.591930 21469 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 19:36:26.616300 21469 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:36:26.619185 21469 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Loaded TSK: 0
I20260926 19:36:26.621558 21469 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 19:36:26.859515 21462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:26.859838 21462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:26.859989 21462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:26.867928 21462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:26.868044 21462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:26.868223 21462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:36:26.878280 21462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:35305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=35479
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:26.879634 21462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:26.881004 21462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:26.895871 21487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:26.895844 21488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:26.896739 21462 server_base.cc:1034] running on GCE node
W20260926 19:36:26.896854 21490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:26.897583 21462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:26.898672 21462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:26.899912 21462 hybrid_clock.cc:648] HybridClock initialized: now 1790451386899849 us; error 61 us; skew 500 ppm
I20260926 19:36:26.900250 21462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:26 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451386, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:26.903537 21462 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:36:26.905128 21462 webserver.cc:460] Webserver started at http://127.6.215.193:35479/ using document root <none> and password file <none>
I20260926 19:36:26.905830 21462 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:26.905983 21462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:26.911559 21462 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 19:36:26.914554 21497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:26.916103 21462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:36:26.916287 21462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "60bf19187c744aceb7bac29400c76aee"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "998a8485c0706d4a220d567a01c9043e"
server_key_iv: "9dcffec87734ca201a41d12b579558d0"
server_key_version: "encryptionkey@0"
I20260926 19:36:26.917390 21462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:26.933772 21462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:26.938406 21462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:26.938701 21462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:26.939836 21462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:26.941344 21462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:26.941478 21462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:26.941597 21462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:26.941685 21462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:26.978155 21462 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:35305
I20260926 19:36:26.978248 21609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:35305 every 8 connection(s)
I20260926 19:36:26.979713 21462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:36:26.984651 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21462
I20260926 19:36:26.985448 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:34299
--local_ip_for_outbound_sockets=127.6.215.194
--tserver_master_addrs=127.6.215.254:42269
--webserver_port=40267
--webserver_interface=127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:36:26 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451386, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:27.005299 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:26.982776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:39347)
0926 19:36:26.983565 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:26.983606 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:26.986079 (+ 2473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:26.987458 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:26.987470 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:26.988241 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:26.989066 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:26.989095 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:26.990709 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:26.990718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:26.992750 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:26.992763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:26.993852 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:26.993860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:26.994147 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:26.994876 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:26.994898 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:26.997383 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:27.000844 (+ 3461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:27.000853 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:27.000871 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:27.001300 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:27.001773 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:27.001778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:27.001781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:27.001971 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:27.002736 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:27.002744 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:27.003088 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 19:36:27.003348 (+ 260us) client_negotiation.cc:240] Negotiation successful
0926 19:36:27.003934 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":374,"threads_started":1}
I20260926 19:36:27.005299 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:26.982742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:39347 (local address 127.6.215.254:42269)
0926 19:36:26.983412 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:26.983427 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:26.983742 (+ 315us) server_negotiation.cc:407] Connection header received
0926 19:36:26.986643 (+ 2901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:26.986655 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:26.986967 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:26.987246 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:26.989295 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:26.990522 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:26.992950 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:26.993600 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:26.993813 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:26.997705 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:26.997713 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:26.997729 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:26.997768 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:27.000647 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.001456 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.001463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.001468 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.001629 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.002105 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.002110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.002112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.002593 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:27.002795 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:27.003576 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 19:36:27.003961 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":250,"threads_started":1}
I20260926 19:36:27.008137 21610 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42269
I20260926 19:36:27.008457 21610 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:27.009217 21610 heartbeater.cc:511] Master 127.6.215.254:42269 requested a full tablet report, sending...
I20260926 19:36:27.011315 21425 ts_manager.cc:194] Registered new tserver with Master: 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305)
I20260926 19:36:27.012727 21425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:39347
W20260926 19:36:27.318828 21614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:27.319159 21614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:27.319329 21614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:27.326848 21614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:27.327045 21614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:27.327235 21614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:36:27.337368 21614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:34299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=40267
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:27.338809 21614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:27.340267 21614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:27.357064 21619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:27.357113 21622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:27.357069 21620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:27.358088 21614 server_base.cc:1034] running on GCE node
I20260926 19:36:27.358731 21614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:27.359745 21614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:27.360962 21614 hybrid_clock.cc:648] HybridClock initialized: now 1790451387360909 us; error 51 us; skew 500 ppm
I20260926 19:36:27.361287 21614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:27 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:27.364574 21614 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:36:27.366338 21614 webserver.cc:460] Webserver started at http://127.6.215.194:40267/ using document root <none> and password file <none>
I20260926 19:36:27.367084 21614 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:27.367246 21614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:27.372596 21614 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:36:27.375389 21629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:27.377090 21614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:27.377282 21614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "26c348a5f9f04f56890369df08809928"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "565ab0ca36dcf283969de700c4590478"
server_key_iv: "3fe7cc226a2012b1fc657fed83e4640e"
server_key_version: "encryptionkey@0"
I20260926 19:36:27.378333 21614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:27.406010 21614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:27.410614 21614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:27.410900 21614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:27.411994 21614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:27.414083 21614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:27.414222 21614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:27.414377 21614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:27.414489 21614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:27.449502 21614 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:34299
I20260926 19:36:27.449568 21741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:34299 every 8 connection(s)
I20260926 19:36:27.450941 21614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:36:27.451280 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21614
I20260926 19:36:27.452360 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:44387
--local_ip_for_outbound_sockets=127.6.215.195
--tserver_master_addrs=127.6.215.254:42269
--webserver_port=42665
--webserver_interface=127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:36:27 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:27.472308 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.454050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:51589 (local address 127.6.215.254:42269)
0926 19:36:27.454234 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:27.454238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:27.455540 (+ 1302us) server_negotiation.cc:407] Connection header received
0926 19:36:27.457522 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:27.457528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:27.457628 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:27.457770 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:27.459357 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.460085 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.461811 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.462255 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.462402 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:27.466476 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:27.466482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:27.466493 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:27.466527 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:27.469239 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.469984 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.469989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.469991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.470154 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.470596 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.470603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.470610 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.470865 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:27.471020 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:27.471714 (+ 694us) server_negotiation.cc:299] Negotiation successful
0926 19:36:27.471946 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:36:27.473565 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.454601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:51589)
0926 19:36:27.455362 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:27.455398 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:27.457227 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:27.458021 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:27.458030 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:27.458572 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:27.459158 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.459176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.460256 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.460263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:27.461638 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.461661 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.462498 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.462505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:27.462761 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:27.463465 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:27.463488 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:27.466257 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:27.469387 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:27.469396 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:27.469414 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:27.469846 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:27.470268 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:27.470273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:27.470277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:27.470469 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:27.470994 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:27.471003 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:27.471346 (+ 343us) client_negotiation.cc:769] Sending connection context
0926 19:36:27.471614 (+ 268us) client_negotiation.cc:240] Negotiation successful
0926 19:36:27.472149 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":331,"threads_started":1}
I20260926 19:36:27.475184 21742 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42269
I20260926 19:36:27.475628 21742 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:27.476579 21742 heartbeater.cc:511] Master 127.6.215.254:42269 requested a full tablet report, sending...
I20260926 19:36:27.478212 21425 ts_manager.cc:194] Registered new tserver with Master: 26c348a5f9f04f56890369df08809928 (127.6.215.194:34299)
I20260926 19:36:27.479351 21425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:51589
W20260926 19:36:27.793586 21745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:27.793965 21745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:27.794145 21745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:27.801828 21745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:27.802006 21745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:27.802201 21745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:36:27.812292 21745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:44387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=42665
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:27.813743 21745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:27.815106 21745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:27.829193 21750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:27.829224 21751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:27.829156 21753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:27.830194 21745 server_base.cc:1034] running on GCE node
I20260926 19:36:27.830817 21745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:27.831753 21745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:27.832998 21745 hybrid_clock.cc:648] HybridClock initialized: now 1790451387832955 us; error 55 us; skew 500 ppm
I20260926 19:36:27.833320 21745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:27 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:27.836444 21745 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:36:27.837973 21745 webserver.cc:460] Webserver started at http://127.6.215.195:42665/ using document root <none> and password file <none>
I20260926 19:36:27.838642 21745 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:27.838753 21745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:27.844259 21745 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 19:36:27.847167 21760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:27.848485 21745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 19:36:27.848629 21745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "659ef73a1d214766836c172c8f5992c3"
format_stamp: "Formatted at 2026-09-26 19:36:25 on dist-test-slave-phbh"
server_key: "1b7fc7a247e7e1dd5bfd16bef09030df"
server_key_iv: "f24e3753fd29654d1c00ef03036ad833"
server_key_version: "encryptionkey@0"
I20260926 19:36:27.849601 21745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:27.869226 21745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:27.873232 21745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:27.873461 21745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:27.874382 21745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:27.875846 21745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:27.875932 21745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:27.876009 21745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:27.876068 21745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:27.910048 21745 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:44387
I20260926 19:36:27.910144 21872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:44387 every 8 connection(s)
I20260926 19:36:27.911595 21745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:36:27.917943 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21745
Sep 26 19:36:27 dist-test-slave-phbh krb5kdc[20840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:27.933019 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.914104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:58197 (local address 127.6.215.254:42269)
0926 19:36:27.914282 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:27.914286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:27.915367 (+ 1081us) server_negotiation.cc:407] Connection header received
0926 19:36:27.917203 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:27.917207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:27.917279 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:27.917367 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:27.919049 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.920035 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.921781 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.922319 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.922457 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:27.926201 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:27.926209 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:27.926217 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:27.926264 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:27.929576 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.930530 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.930534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.930536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.930669 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.931142 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.931145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.931147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.931406 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:27.931582 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:27.932355 (+ 773us) server_negotiation.cc:299] Negotiation successful
0926 19:36:27.932586 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:36:27.934147 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.914522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:58197)
0926 19:36:27.915191 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:27.915233 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:27.916940 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:27.917596 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:27.917603 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:27.918144 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:27.918856 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.918883 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.920212 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.920219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:27.921630 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.921642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.922556 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.922566 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:27.922802 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:27.923466 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:27.923492 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:27.925969 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:27.929713 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:27.929724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:27.929744 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:27.930395 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:27.930783 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:27.930788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:27.930792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:27.931028 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:27.931570 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:27.931576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:27.931919 (+ 343us) client_negotiation.cc:769] Sending connection context
0926 19:36:27.932224 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 19:36:27.932707 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":268,"threads_started":1}
I20260926 19:36:27.934625 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.918877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.6.215.254:42269)
0926 19:36:27.919765 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:27.919770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:27.919794 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:36:27.919832 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:27.919835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:27.919893 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:27.920018 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:27.921833 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.922657 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.924064 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.924577 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.924708 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:27.928280 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:27.928285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:27.928288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:27.928329 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:27.931371 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.932078 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.932082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.932085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.932222 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:27.932665 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:27.932670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:27.932672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:27.933076 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:27.933352 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:27.934018 (+ 666us) server_negotiation.cc:299] Negotiation successful
0926 19:36:27.934252 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":603,"threads_started":1}
I20260926 19:36:27.935771 21873 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42269
I20260926 19:36:27.936113 21873 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:27.936928 21873 heartbeater.cc:511] Master 127.6.215.254:42269 requested a full tablet report, sending...
I20260926 19:36:27.938431 21424 ts_manager.cc:194] Registered new tserver with Master: 659ef73a1d214766836c172c8f5992c3 (127.6.215.195:44387)
I20260926 19:36:27.939285 21424 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:58197
I20260926 19:36:27.949750 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:36:27.956351 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.950553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58480 (local address 127.6.215.254:42269)
0926 19:36:27.950801 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:27.950810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:27.951172 (+ 362us) server_negotiation.cc:407] Connection header received
0926 19:36:27.951340 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:27.951344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:27.951477 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:27.951613 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:36:27.951983 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.952718 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.954374 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.954804 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.954895 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:27.955313 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:36:27.955801 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:36:27.955910 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:27.955989 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 19:36:27.956050 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:36:27.957319 21424 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:36:27.964872 21424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:36:28.001837 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.976153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58350)
0926 19:36:27.977446 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:27.977498 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:27.977708 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:27.994188 (+ 16480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:27.994195 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:27.994224 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:27.994862 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.994883 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.996974 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.996980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:27.999147 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.999160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.000846 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.000854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:28.001039 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.001099 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 19:36:28.001216 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 19:36:28.001399 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"mutex_wait_us":39,"thread_start_us":17402,"threads_started":1}
I20260926 19:36:28.002441 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.977199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56624 (local address 127.6.215.195:44387)
0926 19:36:27.993219 (+ 16020us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:27.993231 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:27.993273 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:36:27.993406 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:27.993419 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:27.993732 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:27.993932 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:27.995139 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.996343 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.000046 (+ 3703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.001240 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.001387 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.001642 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.001951 (+ 309us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.002127 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15779,"thread_start_us":15605,"threads_started":1}
I20260926 19:36:28.002450 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.976510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:44387 (local address 127.0.0.1:56624)
0926 19:36:27.992257 (+ 15747us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:27.992289 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:27.992478 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:27.994228 (+ 1750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:27.994232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:27.994255 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:27.994906 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.994920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.996492 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.996497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:27.999899 (+ 3402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.999909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.001468 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.001474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:28.001738 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.001773 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:36:28.001881 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 19:36:28.002007 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15566,"mutex_wait_us":67,"thread_start_us":948,"threads_started":1}
I20260926 19:36:28.003427 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.978277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:34299 (local address 127.0.0.1:45226)
0926 19:36:27.993820 (+ 15543us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:27.993844 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:27.993966 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:27.994879 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:27.994884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:27.994909 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:27.995347 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.995361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.997269 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.997276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:27.999742 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:27.999757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.002471 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.002480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:28.002843 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.002878 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:36:28.002976 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 19:36:28.003093 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15430,"mutex_wait_us":2,"thread_start_us":13614,"threads_started":1}
I20260926 19:36:28.003557 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.978617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.6.215.194:34299)
0926 19:36:27.990596 (+ 11979us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:27.990603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:27.994115 (+ 3512us) server_negotiation.cc:407] Connection header received
0926 19:36:27.994213 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:27.994220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:27.994424 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:27.994607 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:27.995552 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.997130 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.999909 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.001982 (+ 2073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.002117 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.002297 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.003051 (+ 754us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.003223 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11817,"thread_start_us":11661,"threads_started":1}
I20260926 19:36:28.003557 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:27.976368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.6.215.193:35305)
0926 19:36:27.993566 (+ 17198us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:27.993576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:27.993610 (+ 34us) server_negotiation.cc:407] Connection header received
0926 19:36:27.993708 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:27.993715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:27.993950 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:27.994942 (+ 992us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:27.995193 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:27.996830 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:27.999316 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.000591 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.002791 (+ 2200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.002993 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.003053 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.003223 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17006,"thread_start_us":16866,"threads_started":1}
I20260926 19:36:28.006603 21808 tablet_service.cc:1467] Processing CreateTablet for tablet 90fc84fc12f0413fac66269ce17edacc (DEFAULT_TABLE table=test-table [id=7c7e54f094d74ceaa6e2b4b801edfd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:28.007056 21545 tablet_service.cc:1467] Processing CreateTablet for tablet 90fc84fc12f0413fac66269ce17edacc (DEFAULT_TABLE table=test-table [id=7c7e54f094d74ceaa6e2b4b801edfd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:28.007045 21677 tablet_service.cc:1467] Processing CreateTablet for tablet 90fc84fc12f0413fac66269ce17edacc (DEFAULT_TABLE table=test-table [id=7c7e54f094d74ceaa6e2b4b801edfd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:28.007893 21808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fc84fc12f0413fac66269ce17edacc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:28.008327 21677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fc84fc12f0413fac66269ce17edacc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:28.008351 21545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fc84fc12f0413fac66269ce17edacc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:28.015776 21610 heartbeater.cc:503] Master 127.6.215.254:42269 was elected leader, sending a full tablet report...
I20260926 19:36:28.021376 21889 tablet_bootstrap.cc:492] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Bootstrap starting.
I20260926 19:36:28.024168 21890 tablet_bootstrap.cc:492] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Bootstrap starting.
I20260926 19:36:28.025619 21889 tablet_bootstrap.cc:654] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:28.026739 21889 log.cc:824] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:28.027335 21891 tablet_bootstrap.cc:492] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Bootstrap starting.
I20260926 19:36:28.027905 21890 tablet_bootstrap.cc:654] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:28.029302 21890 log.cc:824] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:28.030978 21889 tablet_bootstrap.cc:492] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: No bootstrap required, opened a new log
I20260926 19:36:28.030961 21891 tablet_bootstrap.cc:654] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:28.031277 21889 ts_tablet_manager.cc:1397] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260926 19:36:28.031857 21890 tablet_bootstrap.cc:492] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: No bootstrap required, opened a new log
I20260926 19:36:28.032115 21890 ts_tablet_manager.cc:1397] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 19:36:28.032202 21891 log.cc:824] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:28.035764 21891 tablet_bootstrap.cc:492] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: No bootstrap required, opened a new log
I20260926 19:36:28.036053 21891 ts_tablet_manager.cc:1397] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260926 19:36:28.039713 21889 raft_consensus.cc:348] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.040103 21889 raft_consensus.cc:374] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:28.040218 21889 raft_consensus.cc:729] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 659ef73a1d214766836c172c8f5992c3, State: Initialized, Role: FOLLOWER
I20260926 19:36:28.040967 21889 consensus_queue.cc:260] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.042076 21873 heartbeater.cc:503] Master 127.6.215.254:42269 was elected leader, sending a full tablet report...
I20260926 19:36:28.042656 21890 raft_consensus.cc:348] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.043220 21890 raft_consensus.cc:374] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:28.043332 21890 raft_consensus.cc:729] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c348a5f9f04f56890369df08809928, State: Initialized, Role: FOLLOWER
I20260926 19:36:28.044338 21889 ts_tablet_manager.cc:1428] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260926 19:36:28.044096 21890 consensus_queue.cc:260] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.045403 21742 heartbeater.cc:503] Master 127.6.215.254:42269 was elected leader, sending a full tablet report...
I20260926 19:36:28.046037 21890 ts_tablet_manager.cc:1428] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260926 19:36:28.046970 21891 raft_consensus.cc:348] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.047433 21891 raft_consensus.cc:374] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:28.047616 21891 raft_consensus.cc:729] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60bf19187c744aceb7bac29400c76aee, State: Initialized, Role: FOLLOWER
I20260926 19:36:28.048290 21891 consensus_queue.cc:260] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.050920 21891 ts_tablet_manager.cc:1428] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260926 19:36:28.068524 21895 raft_consensus.cc:482] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:36:28.068876 21895 raft_consensus.cc:504] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.070499 21895 leader_election.cc:288] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305), 26c348a5f9f04f56890369df08809928 (127.6.215.194:34299)
I20260926 19:36:28.077862 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.070998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:38885 (local address 127.6.215.193:35305)
0926 19:36:28.071214 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:28.071221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:28.071426 (+ 205us) server_negotiation.cc:407] Connection header received
0926 19:36:28.071705 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:28.071711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:28.071787 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:28.071892 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:28.072345 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.073378 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.075762 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.076993 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.077154 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.077229 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.077504 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.077605 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:36:28.077929 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.070992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.6.215.195:38885)
0926 19:36:28.071289 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.071304 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:28.071489 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:28.071913 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:28.071918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:28.071994 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:28.072183 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:28.072192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.073497 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.073500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:28.075614 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:28.075628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.077143 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.077147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:28.077327 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.077373 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 19:36:28.077471 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 19:36:28.077619 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"spinlock_wait_cycles":25472}
I20260926 19:36:28.078157 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.070956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:34855 (local address 127.6.215.194:34299)
0926 19:36:28.071179 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:28.071187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:28.071887 (+ 700us) server_negotiation.cc:407] Connection header received
0926 19:36:28.072097 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:28.072101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:28.072156 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:28.072225 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:28.072750 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.073711 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.075965 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.077225 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.077352 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.077398 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.077742 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.077869 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 19:36:28.078403 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.071047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:34299 (local address 127.6.215.195:34855)
0926 19:36:28.071783 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.071795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:28.071941 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:28.072267 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:28.072273 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:28.072298 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:28.072594 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:28.072601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.073845 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.073852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:28.075832 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:28.075841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.077408 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.077416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:28.077584 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.077613 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 19:36:28.077695 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 19:36:28.077815 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":416,"threads_started":1}
I20260926 19:36:28.078980 21565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fc84fc12f0413fac66269ce17edacc" candidate_uuid: "659ef73a1d214766836c172c8f5992c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60bf19187c744aceb7bac29400c76aee" is_pre_election: true
I20260926 19:36:28.079550 21565 raft_consensus.cc:2454] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 659ef73a1d214766836c172c8f5992c3 in term 0.
I20260926 19:36:28.079559 21697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fc84fc12f0413fac66269ce17edacc" candidate_uuid: "659ef73a1d214766836c172c8f5992c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26c348a5f9f04f56890369df08809928" is_pre_election: true
I20260926 19:36:28.080080 21697 raft_consensus.cc:2454] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 659ef73a1d214766836c172c8f5992c3 in term 0.
I20260926 19:36:28.080309 21764 leader_election.cc:302] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60bf19187c744aceb7bac29400c76aee, 659ef73a1d214766836c172c8f5992c3; no voters:
I20260926 19:36:28.080732 21895 raft_consensus.cc:2781] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:36:28.080933 21895 raft_consensus.cc:482] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:36:28.081061 21895 raft_consensus.cc:3037] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:28.083387 21895 raft_consensus.cc:504] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.083983 21895 leader_election.cc:288] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [CANDIDATE]: Term 1 election: Requested vote from peers 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305), 26c348a5f9f04f56890369df08809928 (127.6.215.194:34299)
I20260926 19:36:28.084455 21565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fc84fc12f0413fac66269ce17edacc" candidate_uuid: "659ef73a1d214766836c172c8f5992c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60bf19187c744aceb7bac29400c76aee"
I20260926 19:36:28.084497 21697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fc84fc12f0413fac66269ce17edacc" candidate_uuid: "659ef73a1d214766836c172c8f5992c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26c348a5f9f04f56890369df08809928"
I20260926 19:36:28.084686 21697 raft_consensus.cc:3037] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:28.084686 21565 raft_consensus.cc:3037] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:28.087853 21565 raft_consensus.cc:2454] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 659ef73a1d214766836c172c8f5992c3 in term 1.
I20260926 19:36:28.087853 21697 raft_consensus.cc:2454] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 659ef73a1d214766836c172c8f5992c3 in term 1.
I20260926 19:36:28.088282 21762 leader_election.cc:302] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c348a5f9f04f56890369df08809928, 659ef73a1d214766836c172c8f5992c3; no voters:
I20260926 19:36:28.088629 21895 raft_consensus.cc:2781] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:28.089114 21895 raft_consensus.cc:686] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 1 LEADER]: Becoming Leader. State: Replica: 659ef73a1d214766836c172c8f5992c3, State: Running, Role: LEADER
I20260926 19:36:28.089576 21895 consensus_queue.cc:237] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } }
I20260926 19:36:28.094300 21424 catalog_manager.cc:5495] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 659ef73a1d214766836c172c8f5992c3 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "659ef73a1d214766836c172c8f5992c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "659ef73a1d214766836c172c8f5992c3" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 } health_report { overall_health: UNKNOWN } } }
I20260926 19:36:28.131805 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.126195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56630 (local address 127.6.215.195:44387)
0926 19:36:28.126415 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:28.126420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:28.126441 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:36:28.126658 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:28.126663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:28.126742 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:28.126903 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:36:28.127284 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.128027 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.129661 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.130068 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.130209 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.130724 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:36:28.131230 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:36:28.131403 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.131464 (+ 61us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.131542 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:36:28.140165 21565 raft_consensus.cc:1261] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Refusing update from remote peer 659ef73a1d214766836c172c8f5992c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:28.140164 21697 raft_consensus.cc:1261] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 1 FOLLOWER]: Refusing update from remote peer 659ef73a1d214766836c172c8f5992c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:28.141248 21899 consensus_queue.cc:1035] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c348a5f9f04f56890369df08809928" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 34299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:28.141593 21895 consensus_queue.cc:1035] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60bf19187c744aceb7bac29400c76aee" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 35305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:28.153646 21902 mvcc.cc:205] Tried to move back new op lower bound from 7333688885811970048 to 7333688885619163136. Current Snapshot: MvccSnapshot[applied={T|T < 7333688885811970048}]
I20260926 19:36:28.167455 21425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:28.167771 21425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:28.171208 21425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Sending DeleteTablet for 3 replicas of tablet 90fc84fc12f0413fac66269ce17edacc
I20260926 19:36:28.172465 21808 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fc84fc12f0413fac66269ce17edacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:28 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:56624
I20260926 19:36:28.172485 21545 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fc84fc12f0413fac66269ce17edacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:28 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:58350
I20260926 19:36:28.172644 21677 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fc84fc12f0413fac66269ce17edacc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:28 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:45226
I20260926 19:36:28.173421 21912 tablet_replica.cc:330] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: stopping tablet replica
I20260926 19:36:28.173920 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 21462
I20260926 19:36:28.173898 21912 raft_consensus.cc:2229] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:36:28.175083 21912 raft_consensus.cc:2258] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:28.179178 21912 ts_tablet_manager.cc:1905] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:28.187144 21913 tablet_replica.cc:330] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: stopping tablet replica
I20260926 19:36:28.187717 21913 raft_consensus.cc:2229] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:28.187837 21914 tablet_replica.cc:330] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: stopping tablet replica
I20260926 19:36:28.188231 21914 raft_consensus.cc:2229] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:28.188309 21913 raft_consensus.cc:2258] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:28.188705 21914 raft_consensus.cc:2258] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:28.190657 21914 ts_tablet_manager.cc:1905] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:28.190665 21913 ts_tablet_manager.cc:1905] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:28.192729 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.174841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.6.215.193:35305)
0926 19:36:28.175005 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:28.175012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:28.175149 (+ 137us) server_negotiation.cc:407] Connection header received
0926 19:36:28.175345 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:28.175350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:28.175432 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:28.175544 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:28.177387 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.178628 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.180361 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.181105 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.181232 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.184795 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:28.184813 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:28.184833 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:28.184882 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:28.189542 (+ 4660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:28.190432 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:28.190441 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:28.190444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:28.190613 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:28.191003 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:28.191009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:28.191012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:28.191485 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:28.191829 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.192101 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.192326 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 19:36:28.193679 21605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:28.196066 21912 ts_tablet_manager.cc:1918] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:28.196272 21912 log.cc:1196] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/90fc84fc12f0413fac66269ce17edacc
I20260926 19:36:28.196941 21912 ts_tablet_manager.cc:1939] T 90fc84fc12f0413fac66269ce17edacc P 659ef73a1d214766836c172c8f5992c3: Deleting consensus metadata
I20260926 19:36:28.198942 21410 catalog_manager.cc:4841] TS 659ef73a1d214766836c172c8f5992c3 (127.6.215.195:44387): tablet 90fc84fc12f0413fac66269ce17edacc (table test-table [id=7c7e54f094d74ceaa6e2b4b801edfd76]) successfully deleted
I20260926 19:36:28.204420 21914 ts_tablet_manager.cc:1918] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:28.204619 21914 log.cc:1196] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/90fc84fc12f0413fac66269ce17edacc
I20260926 19:36:28.205606 21914 ts_tablet_manager.cc:1939] T 90fc84fc12f0413fac66269ce17edacc P 26c348a5f9f04f56890369df08809928: Deleting consensus metadata
I20260926 19:36:28.207960 21410 catalog_manager.cc:4841] TS 26c348a5f9f04f56890369df08809928 (127.6.215.194:34299): tablet 90fc84fc12f0413fac66269ce17edacc (table test-table [id=7c7e54f094d74ceaa6e2b4b801edfd76]) successfully deleted
I20260926 19:36:28.328045 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 21462
I20260926 19:36:28.346295 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 21614
W20260926 19:36:28.346475 21412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.215.193:35305 (error 108)
I20260926 19:36:28.348425 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.347918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58370)
0926 19:36:28.348095 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.348217 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 19:36:28.348698 21412 catalog_manager.cc:4584] TS 60bf19187c744aceb7bac29400c76aee (127.6.215.193:35305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90fc84fc12f0413fac66269ce17edacc: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
I20260926 19:36:28.363806 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.347268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45236 (local address 127.6.215.194:34299)
0926 19:36:28.347467 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:28.347475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:28.348127 (+ 652us) server_negotiation.cc:407] Connection header received
0926 19:36:28.348208 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:28.348214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:28.348307 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:28.348430 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:28.350649 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.351399 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.353613 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.354009 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.354191 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.357986 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:28.358001 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:28.358015 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:28.358062 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:28.360946 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:28.361834 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:28.361840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:28.361843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:28.361959 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:28.362391 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:28.362394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:28.362396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:28.362758 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:28.362979 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.363364 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.363524 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 19:36:28.364542 21737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:28.402557 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.401956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58372)
0926 19:36:28.402208 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.402346 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 19:36:28.403791 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.403373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58374)
0926 19:36:28.403546 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.403645 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 19:36:28.456033 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 21614
I20260926 19:36:28.464591 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.464006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58386)
0926 19:36:28.464218 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.464314 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 19:36:28.465888 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.465399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58394)
0926 19:36:28.465619 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.465708 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260926 19:36:28.484157 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 21745
I20260926 19:36:28.501729 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.484996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.6.215.195:44387)
0926 19:36:28.485182 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:28.485188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:28.485399 (+ 211us) server_negotiation.cc:407] Connection header received
0926 19:36:28.485622 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:28.485627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:28.485708 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:28.485802 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:28.487790 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.489235 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.491199 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:28.491645 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:28.491742 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:28.495191 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:28.495202 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:28.495216 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:28.495256 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:28.498429 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:28.499419 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:28.499426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:28.499428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:28.499582 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:28.500017 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:28.500020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:28.500022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:28.500408 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:28.500635 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:28.501215 (+ 580us) server_negotiation.cc:299] Negotiation successful
0926 19:36:28.501386 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:36:28.502456 21868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:28.547183 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.546680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58406)
0926 19:36:28.546889 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.547003 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 19:36:28.548319 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.547929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58420)
0926 19:36:28.548141 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.548210 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260926 19:36:28.626796 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 21745
I20260926 19:36:28.647759 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 21392
I20260926 19:36:28.648941 21455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:28.763315 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:28.762638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:35305 (local address 127.0.0.1:58430)
0926 19:36:28.762982 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:28.763105 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:35305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260926 19:36:28.763291 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 21392
I20260926 19:36:28.785377 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42269
--webserver_interface=127.6.215.254
--webserver_port=37115
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:29.121395 21920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:29.121752 21920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:29.121906 21920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:29.129263 21920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:29.129426 21920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:29.129534 21920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:29.129618 21920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:29.129698 21920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:29.139662 21920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=37115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:29.141140 21920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:29.142485 21920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:29.154440 21926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:29.154563 21928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:29.155217 21925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:29.156497 21920 server_base.cc:1034] running on GCE node
I20260926 19:36:29.157398 21920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:29.158885 21920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:29.160097 21920 hybrid_clock.cc:648] HybridClock initialized: now 1790451389160041 us; error 47 us; skew 500 ppm
I20260926 19:36:29.160475 21920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:29 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451389, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:29.170056 21920 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:29.171816 21920 webserver.cc:460] Webserver started at http://127.6.215.254:37115/ using document root <none> and password file <none>
I20260926 19:36:29.172564 21920 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:29.172714 21920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:29.178447 21920 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 19:36:29.181595 21935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:29.182948 21920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:29.183148 21920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2f1746b667c046d183e53e9b91b979e5"
format_stamp: "Formatted at 2026-09-26 19:36:23 on dist-test-slave-phbh"
server_key: "9aa81ae6055e7c594522b22c3301a2b7"
server_key_iv: "70c3ad0a8143a46415b1bc77859ef59b"
server_key_version: "encryptionkey@0"
I20260926 19:36:29.184181 21920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:29.203905 21920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:29.208139 21920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:29.208429 21920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:29.227129 21920 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42269
I20260926 19:36:29.227149 21986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42269 every 8 connection(s)
I20260926 19:36:29.228721 21920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:29.235632 21987 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:36:29.238019 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21920
I20260926 19:36:29.238858 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:35305
--local_ip_for_outbound_sockets=127.6.215.193
--tserver_master_addrs=127.6.215.254:42269
--webserver_port=35479
--webserver_interface=127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:36:29.238904 21987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Bootstrap starting.
I20260926 19:36:29.260975 21987 log.cc:824] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:29.277767 21987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:36:29.278350 21987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5: Bootstrap complete.
I20260926 19:36:29.287313 21987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:29.288347 21987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f1746b667c046d183e53e9b91b979e5, State: Initialized, Role: FOLLOWER
I20260926 19:36:29.289081 21987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:29.289332 21987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:36:29.289551 21987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:36:29.289752 21987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:36:29.293634 21987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:29.294139 21987 leader_election.cc:302] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f1746b667c046d183e53e9b91b979e5; no voters:
I20260926 19:36:29.294818 21987 leader_election.cc:288] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 19:36:29.295014 21991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 19:36:29.296183 21991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [term 3 LEADER]: Becoming Leader. State: Replica: 2f1746b667c046d183e53e9b91b979e5, State: Running, Role: LEADER
I20260926 19:36:29.296813 21991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } }
I20260926 19:36:29.297142 21987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:29.299348 21993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f1746b667c046d183e53e9b91b979e5. Latest consensus state: current_term: 3 leader_uuid: "2f1746b667c046d183e53e9b91b979e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } } }
I20260926 19:36:29.299700 21993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:29.300285 21992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2f1746b667c046d183e53e9b91b979e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f1746b667c046d183e53e9b91b979e5" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42269 } } }
I20260926 19:36:29.300552 21992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f1746b667c046d183e53e9b91b979e5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:29.310988 21996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:29.323974 21996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:29.326023 21996 catalog_manager.cc:1259] Loaded cluster ID: 59f997454537456d839ef083edcfa486
I20260926 19:36:29.326156 21996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 19:36:29.329306 21996 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb112e469ad google::LogMessage::Fail() at ??:0
@ 0x7fb112e4988a google::LogMessage::SendToLog() at ??:0
@ 0x7fb112e464de google::LogMessage::Flush() at ??:0
@ 0x7fb112e47a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb128879512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb128876976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb1288fe610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb1288fe439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb115a92345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb115a9591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb115a95799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb115a70eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb118d4a6db start_thread at ??:0
@ 0x7fb111ce061f clone at ??:0
W20260926 19:36:29.627526 21989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:29.627969 21989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:29.628147 21989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:29.640003 21989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:29.640136 21989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:29.640312 21989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:36:29.658169 21989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:35305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=35479
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:29.659741 21989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:29.661145 21989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:29.674198 22028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:29.674270 22031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:29.674242 22029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:29.675390 21989 server_base.cc:1034] running on GCE node
I20260926 19:36:29.676067 21989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:29.677153 21989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:29.678356 21989 hybrid_clock.cc:648] HybridClock initialized: now 1790451389678314 us; error 27 us; skew 500 ppm
I20260926 19:36:29.678666 21989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:29 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451389, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:29.681672 21989 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:36:29.683116 21989 webserver.cc:460] Webserver started at http://127.6.215.193:35479/ using document root <none> and password file <none>
I20260926 19:36:29.683830 21989 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:29.683991 21989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:29.689333 21989 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:36:29.692284 22038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:29.693712 21989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:29.693902 21989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "60bf19187c744aceb7bac29400c76aee"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "998a8485c0706d4a220d567a01c9043e"
server_key_iv: "9dcffec87734ca201a41d12b579558d0"
server_key_version: "encryptionkey@0"
I20260926 19:36:29.694881 21989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:29.711845 21989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:29.715797 21989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:29.716077 21989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:29.717105 21989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:29.718879 22045 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260926 19:36:29.724037 22045 ts_tablet_manager.cc:1852] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260926 19:36:29.725023 22045 ts_tablet_manager.cc:1905] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:29.731992 22045 ts_tablet_manager.cc:1918] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:29.732191 22045 log.cc:1196] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/90fc84fc12f0413fac66269ce17edacc
I20260926 19:36:29.732683 22045 ts_tablet_manager.cc:1939] T 90fc84fc12f0413fac66269ce17edacc P 60bf19187c744aceb7bac29400c76aee: Deleting consensus metadata
I20260926 19:36:29.733079 21989 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260926 19:36:29.733220 21989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260926 19:36:29.733361 21989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:29.733451 21989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:29.767395 21989 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:35305
I20260926 19:36:29.767484 22151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:35305 every 8 connection(s)
I20260926 19:36:29.768889 21989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:36:29.773883 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:29.772060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:58553)
0926 19:36:29.772763 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:29.773070 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":279,"threads_started":1}
W20260926 19:36:29.774492 22152 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:42269 (0 consecutive failures): Network error: Failed to ping master at 127.6.215.254:42269: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
I20260926 19:36:29.775312 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:29.774973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:39889)
0926 19:36:29.775135 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:29.775216 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 19:36:29.776146 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:29.775810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:58923)
0926 19:36:29.775946 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:29.776030 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 19:36:29.776376 22152 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 19:36:29.779012 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 21989
I20260926 19:36:29.779695 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:34299
--local_ip_for_outbound_sockets=127.6.215.194
--tserver_master_addrs=127.6.215.254:42269
--webserver_port=40267
--webserver_interface=127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:30.112599 22155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:30.112995 22155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:30.113224 22155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:30.120683 22155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:30.120847 22155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:30.121057 22155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:36:30.130790 22155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:34299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=40267
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:30.132198 22155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:30.133620 22155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:30.147404 22160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:30.147486 22163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:30.147511 22161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:30.148579 22155 server_base.cc:1034] running on GCE node
I20260926 19:36:30.149307 22155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:30.150223 22155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:30.151423 22155 hybrid_clock.cc:648] HybridClock initialized: now 1790451390151370 us; error 40 us; skew 500 ppm
I20260926 19:36:30.151768 22155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:30 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451390, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:30.154942 22155 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:36:30.156489 22155 webserver.cc:460] Webserver started at http://127.6.215.194:40267/ using document root <none> and password file <none>
I20260926 19:36:30.157255 22155 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:30.157401 22155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:30.162792 22155 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 19:36:30.165573 22170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:30.166924 22155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:30.167114 22155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "26c348a5f9f04f56890369df08809928"
format_stamp: "Formatted at 2026-09-26 19:36:24 on dist-test-slave-phbh"
server_key: "565ab0ca36dcf283969de700c4590478"
server_key_iv: "3fe7cc226a2012b1fc657fed83e4640e"
server_key_version: "encryptionkey@0"
I20260926 19:36:30.168119 22155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:30.183269 22155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:30.187407 22155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:30.187698 22155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:30.188668 22155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:30.189986 22155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:30.190119 22155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:30.190254 22155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:30.190346 22155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:30.224187 22155 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:34299
I20260926 19:36:30.224275 22282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:34299 every 8 connection(s)
I20260926 19:36:30.225679 22155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:36:30.230291 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:30.228503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:36681)
0926 19:36:30.229264 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:30.229477 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":335,"threads_started":1}
W20260926 19:36:30.230978 22283 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:42269 (0 consecutive failures): Network error: Failed to ping master at 127.6.215.254:42269: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
I20260926 19:36:30.231771 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:30.231439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:38187)
0926 19:36:30.231597 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:30.231678 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 19:36:30.232615 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:30.232329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:49831)
0926 19:36:30.232441 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:30.232508 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 19:36:30.232901 22283 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 19:36:30.233141 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 22155
I20260926 19:36:30.233770 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:44387
--local_ip_for_outbound_sockets=127.6.215.195
--tserver_master_addrs=127.6.215.254:42269
--webserver_port=42665
--webserver_interface=127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:30.565649 22286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:30.565914 22286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:30.566111 22286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:30.573830 22286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:30.573987 22286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:30.574185 22286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:36:30.584712 22286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:44387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=42665
--tserver_master_addrs=127.6.215.254:42269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:30.586138 22286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:30.587502 22286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:30.600955 22292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:30.601167 22294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:30.601320 22291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:30.602030 22286 server_base.cc:1034] running on GCE node
I20260926 19:36:30.602773 22286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:30.604143 22286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:30.605408 22286 hybrid_clock.cc:648] HybridClock initialized: now 1790451390605328 us; error 63 us; skew 500 ppm
I20260926 19:36:30.605835 22286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:30 dist-test-slave-phbh krb5kdc[20840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451390, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:30.609256 22286 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:36:30.610793 22286 webserver.cc:460] Webserver started at http://127.6.215.195:42665/ using document root <none> and password file <none>
I20260926 19:36:30.611495 22286 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:30.611634 22286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:30.617410 22286 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260926 19:36:30.620316 22301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:30.621896 22286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:30.622073 22286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "659ef73a1d214766836c172c8f5992c3"
format_stamp: "Formatted at 2026-09-26 19:36:25 on dist-test-slave-phbh"
server_key: "1b7fc7a247e7e1dd5bfd16bef09030df"
server_key_iv: "f24e3753fd29654d1c00ef03036ad833"
server_key_version: "encryptionkey@0"
I20260926 19:36:30.623072 22286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:30.638859 22286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:30.642979 22286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:30.643272 22286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:30.644239 22286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:30.645776 22286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:30.645913 22286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:30.646009 22286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:30.646073 22286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:30.681602 22286 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:44387
I20260926 19:36:30.681684 22413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:44387 every 8 connection(s)
I20260926 19:36:30.683054 22286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:36:30.686007 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 22286
I20260926 19:36:30.688117 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:30.685952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:52757)
0926 19:36:30.686804 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:30.687022 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":334,"threads_started":1}
W20260926 19:36:30.688868 22414 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:42269 (0 consecutive failures): Network error: Failed to ping master at 127.6.215.254:42269: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
I20260926 19:36:30.689626 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:30.689314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:58627)
0926 19:36:30.689472 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:30.689532 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 19:36:30.690438 22416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:30.690126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:41971)
0926 19:36:30.690261 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:30.690337 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 19:36:30.690678 22414 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 19:36:30.778038 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:30.777122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:59943)
0926 19:36:30.777649 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:30.777771 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":275,"threads_started":1}
I20260926 19:36:31.234251 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:31.233437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:47739)
0926 19:36:31.233919 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:31.234028 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260926 19:36:31.692207 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:31.691339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:54413)
0926 19:36:31.691837 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:31.691949 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":256,"threads_started":1}
I20260926 19:36:31.779775 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:31.779033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:39467)
0926 19:36:31.779513 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:31.779612 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":259,"threads_started":1}
I20260926 19:36:32.235939 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:32.235242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:38673)
0926 19:36:32.235678 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:32.235756 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":203,"threads_started":1}
I20260926 19:36:32.693928 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:32.693147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:58169)
0926 19:36:32.693683 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:32.693762 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":297,"threads_started":1}
I20260926 19:36:32.781829 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:32.781069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:52133)
0926 19:36:32.781554 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:32.781650 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":243,"threads_started":1}
I20260926 19:36:33.237784 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:33.237061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:52571)
0926 19:36:33.237508 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:33.237589 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":227,"threads_started":1}
I20260926 19:36:33.695539 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:33.694832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:37683)
0926 19:36:33.695271 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:33.695367 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":225,"threads_started":1}
I20260926 19:36:33.783530 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:33.782717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:39999)
0926 19:36:33.783288 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:33.783368 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":301,"threads_started":1}
I20260926 19:36:34.239591 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:34.238745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:45127)
0926 19:36:34.239269 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:34.239393 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":247,"threads_started":1}
I20260926 19:36:34.697311 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:34.696473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:40337)
0926 19:36:34.696992 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:34.697077 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":223,"threads_started":1}
I20260926 19:36:34.785250 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:34.784483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:45005)
0926 19:36:34.784992 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:34.785080 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":228,"threads_started":1}
I20260926 19:36:35.241415 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:35.240547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:55473)
0926 19:36:35.241138 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:35.241229 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":325,"threads_started":1}
I20260926 19:36:35.699134 22432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:35.698281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:48365)
0926 19:36:35.698829 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:35.698907 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":319,"threads_started":1}
I20260926 19:36:35.786922 22433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:35.786203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:37529)
0926 19:36:35.786664 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:35.786753 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":230,"threads_started":1}
I20260926 19:36:36.243216 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:36.242448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:54027)
0926 19:36:36.242939 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:36.243033 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":254,"threads_started":1}
I20260926 19:36:36.700847 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:36.700029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:46465)
0926 19:36:36.700515 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:36.700613 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":260,"threads_started":1}
I20260926 19:36:36.788841 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:36.787988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:45477)
0926 19:36:36.788509 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:36.788609 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":238,"threads_started":1}
I20260926 19:36:37.245404 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:37.244298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:33465)
0926 19:36:37.244902 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:37.245001 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":214,"threads_started":1}
I20260926 19:36:37.702522 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:37.701723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:33675)
0926 19:36:37.702211 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:37.702304 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":218,"threads_started":1}
I20260926 19:36:37.790550 22439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:37.789799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:56067)
0926 19:36:37.790290 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:37.790389 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":232,"threads_started":1}
I20260926 19:36:38.247138 22440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:38.246391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:35843)
0926 19:36:38.246876 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:38.246970 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":275,"threads_started":1}
I20260926 19:36:38.704238 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:38.703459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:49431)
0926 19:36:38.703977 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:38.704068 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":268,"threads_started":1}
I20260926 19:36:38.792183 22442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:38.791407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:53601)
0926 19:36:38.791896 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:38.792003 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":237,"threads_started":1}
I20260926 19:36:39.248987 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:39.248146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:58891)
0926 19:36:39.248638 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:39.248726 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":233,"threads_started":1}
I20260926 19:36:39.706502 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:39.705554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:49521)
0926 19:36:39.706175 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:39.706279 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":329,"threads_started":1}
I20260926 19:36:39.793962 22445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:39.793281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:54275)
0926 19:36:39.793718 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:39.793800 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":238,"threads_started":1}
I20260926 19:36:40.251017 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:40.250111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:58125)
0926 19:36:40.250657 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:40.250784 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":330,"threads_started":1}
I20260926 19:36:40.710553 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:40.709680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:46803)
0926 19:36:40.710167 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:40.710337 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":278,"threads_started":1}
I20260926 19:36:40.795660 22448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:40.794919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:54305)
0926 19:36:40.795389 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:40.795472 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":264,"threads_started":1}
I20260926 19:36:41.252739 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:41.252021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:48109)
0926 19:36:41.252464 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:41.252561 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":210,"threads_started":1}
I20260926 19:36:41.712350 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:41.711561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:48145)
0926 19:36:41.712042 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:41.712132 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":270,"threads_started":1}
I20260926 19:36:41.797487 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:41.796683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:39615)
0926 19:36:41.797224 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:41.797329 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":340,"threads_started":1}
I20260926 19:36:42.254411 22452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:42.253715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:53077)
0926 19:36:42.254172 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:42.254259 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260926 19:36:42.714069 22453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:42.713362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:36909)
0926 19:36:42.713802 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:42.713909 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":231,"threads_started":1}
I20260926 19:36:42.799335 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:42.798574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:48385)
0926 19:36:42.799043 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:42.799150 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":242,"threads_started":1}
I20260926 19:36:43.256229 22455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:43.255436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:46301)
0926 19:36:43.255936 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:43.256043 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":252,"threads_started":1}
I20260926 19:36:43.715647 22456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:43.714918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:40227)
0926 19:36:43.715376 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:43.715464 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":228,"threads_started":1}
I20260926 19:36:43.801245 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:43.800438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:51667)
0926 19:36:43.800923 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:43.801053 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":214,"threads_started":1}
I20260926 19:36:44.258356 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:44.257394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:37357)
0926 19:36:44.258024 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:44.258129 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":291,"threads_started":1}
I20260926 19:36:44.717417 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:44.716586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:53103)
0926 19:36:44.717121 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:44.717217 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":282,"threads_started":1}
I20260926 19:36:44.803016 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:44.802298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:55481)
0926 19:36:44.802750 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:44.802838 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260926 19:36:45.260206 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:45.259452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.194:43391)
0926 19:36:45.259930 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:45.260016 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":260,"threads_started":1}
I20260926 19:36:45.694088 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 21989
I20260926 19:36:45.719564 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:45.718690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.195:43071)
0926 19:36:45.719203 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:45.719297 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":276,"threads_started":1}
I20260926 19:36:45.722203 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:45.695257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.6.215.193:35305)
0926 19:36:45.696209 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:45.696232 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:45.696294 (+ 62us) server_negotiation.cc:407] Connection header received
0926 19:36:45.698850 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:45.698862 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:45.699455 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:45.699841 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:45.701672 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:45.703082 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:45.704727 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:45.705648 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:45.705898 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:45.714782 (+ 8884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:45.714792 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:45.714812 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:45.714862 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:45.718138 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:45.719073 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:45.719083 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:45.719093 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:45.719311 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:45.719870 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:45.719875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:45.719879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:45.720552 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:45.720864 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:45.721448 (+ 584us) server_negotiation.cc:299] Negotiation successful
0926 19:36:45.721926 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":563,"threads_started":1}
I20260926 19:36:45.723809 22147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:45.812260 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:45.811019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42269 (local address 127.6.215.193:53981)
0926 19:36:45.811628 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:45.811876 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":303,"threads_started":1}
I20260926 19:36:45.812381 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 21989
I20260926 19:36:45.830479 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 22155
I20260926 19:36:45.849907 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:45.831418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.6.215.194:34299)
0926 19:36:45.832026 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:45.832040 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:45.832078 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:36:45.833871 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:45.833881 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:45.834321 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:45.834692 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:45.836311 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:45.837756 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:45.839179 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:45.839890 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:45.840123 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:45.842984 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:45.842993 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:45.843008 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:45.843047 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:45.846263 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:45.847066 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:45.847074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:45.847081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:45.847267 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:45.847698 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:45.847710 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:45.847714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:45.848262 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:45.848615 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:45.849154 (+ 539us) server_negotiation.cc:299] Negotiation successful
0926 19:36:45.849597 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":294,"threads_started":1}
I20260926 19:36:45.851336 22278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:45.936128 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 22155
I20260926 19:36:45.954599 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 22286
I20260926 19:36:45.973654 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:45.955777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.6.215.195:44387)
0926 19:36:45.956451 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:45.956466 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:45.956510 (+ 44us) server_negotiation.cc:407] Connection header received
0926 19:36:45.958418 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:45.958429 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:45.958902 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:45.959229 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:45.960742 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:45.962234 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:45.963494 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:45.964219 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:45.964373 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:45.967154 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:45.967163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:45.967179 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:45.967222 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:45.970359 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:45.971068 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:45.971074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:45.971081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:45.971265 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:45.971641 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:45.971644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:45.971646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:45.972144 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:45.972383 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:45.972889 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 19:36:45.973359 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":330,"threads_started":1}
I20260926 19:36:45.975041 22409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:46.062330 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 22286
2026-09-26T19:36:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24281 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:36:46 dist-test-slave-phbh krb5kdc[22472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:36:46 dist-test-slave-phbh krb5kdc[22472](info): set up 2 sockets
Sep 26 19:36:46 dist-test-slave-phbh krb5kdc[22472](info): commencing operation
krb5kdc: starting...
W20260926 19:36:47.161527 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:36:47 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:36:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:36:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:36:47.409026 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42151
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:47.759965 22485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:47.760245 22485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:47.760418 22485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:47.767660 22485 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 19:36:47.767759 22485 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 19:36:47.767830 22485 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 19:36:47.767907 22485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:36:47.767995 22485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:47.768124 22485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:47.768178 22485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:36:47.768245 22485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:36:47.778154 22485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:47.779481 22485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:47.780864 22485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:47.793337 22490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:47.793478 22491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:47.793670 22485 server_base.cc:1034] running on GCE node
W20260926 19:36:47.793421 22493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:47.794783 22485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:47.796216 22485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:47.797524 22485 hybrid_clock.cc:648] HybridClock initialized: now 1790451407797450 us; error 46 us; skew 500 ppm
I20260926 19:36:47.797900 22485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:47 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451407, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:47.801180 22485 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:36:47.802686 22485 webserver.cc:460] Webserver started at http://127.6.215.254:44071/ using document root <none> and password file <none>
I20260926 19:36:47.803426 22485 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:47.803552 22485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:47.803874 22485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:47.806947 22485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "2107f7ea91214f7698bf62e3c85274ff"
format_stamp: "Formatted at 2026-09-26 19:36:47 on dist-test-slave-phbh"
server_key: "7663e621ce5e2930b66a22ed4918d18e"
server_key_iv: "a54a72a46fe8be5db52918c4b5eb1528"
server_key_version: "encryptionkey@0"
I20260926 19:36:47.807791 22485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "2107f7ea91214f7698bf62e3c85274ff"
format_stamp: "Formatted at 2026-09-26 19:36:47 on dist-test-slave-phbh"
server_key: "7663e621ce5e2930b66a22ed4918d18e"
server_key_iv: "a54a72a46fe8be5db52918c4b5eb1528"
server_key_version: "encryptionkey@0"
I20260926 19:36:47.812930 22485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 19:36:47.816550 22500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:47.818017 22485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:36:47.818200 22485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2107f7ea91214f7698bf62e3c85274ff"
format_stamp: "Formatted at 2026-09-26 19:36:47 on dist-test-slave-phbh"
server_key: "7663e621ce5e2930b66a22ed4918d18e"
server_key_iv: "a54a72a46fe8be5db52918c4b5eb1528"
server_key_version: "encryptionkey@0"
I20260926 19:36:47.818439 22485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:47.833354 22485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:47.837535 22485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:47.837813 22485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:47.857082 22485 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42151
I20260926 19:36:47.857105 22551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42151 every 8 connection(s)
I20260926 19:36:47.858544 22485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:36:47.862607 22552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:47.864650 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 22485
I20260926 19:36:47.864907 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:36:47.865760 7007 external_mini_cluster.cc:1428] Setting key 5c49cc0be474031a9c4008c76332fba4
I20260926 19:36:47.876145 22552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Bootstrap starting.
I20260926 19:36:47.880378 22552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Neither blocks nor log segments found. Creating new log.
Sep 26 19:36:47 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:47.881681 22552 log.cc:824] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:47.884954 22552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: No bootstrap required, opened a new log
I20260926 19:36:47.890199 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:47.868038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46808 (local address 127.6.215.254:42151)
0926 19:36:47.868941 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:47.868954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:47.868999 (+ 45us) server_negotiation.cc:407] Connection header received
0926 19:36:47.870706 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:47.870734 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:47.871143 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:47.871474 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:47.873396 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:47.874765 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:47.876705 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:47.877452 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:47.877593 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:47.882638 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:47.882646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:47.882661 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:47.882701 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:47.885980 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:47.886749 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:47.886756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:47.886762 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:47.886945 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:47.887339 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:47.887344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:47.887349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:47.887902 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:47.888248 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:47.888692 (+ 444us) server_negotiation.cc:299] Negotiation successful
0926 19:36:47.889174 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":387,"threads_started":1}
I20260926 19:36:47.893711 22552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:36:47.894119 22552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:47.894245 22552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2107f7ea91214f7698bf62e3c85274ff, State: Initialized, Role: FOLLOWER
I20260926 19:36:47.894881 22552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:36:47.895133 22552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:36:47.895272 22552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:36:47.895452 22552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:47.898072 22552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:36:47.898619 22552 leader_election.cc:302] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2107f7ea91214f7698bf62e3c85274ff; no voters:
I20260926 19:36:47.899284 22552 leader_election.cc:288] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:36:47.899422 22557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:47.900673 22557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 LEADER]: Becoming Leader. State: Replica: 2107f7ea91214f7698bf62e3c85274ff, State: Running, Role: LEADER
I20260926 19:36:47.901244 22557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:36:47.901628 22552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:36:47.904558 22559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2107f7ea91214f7698bf62e3c85274ff. Latest consensus state: current_term: 1 leader_uuid: "2107f7ea91214f7698bf62e3c85274ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } } }
I20260926 19:36:47.909263 22559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:47.904814 22558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2107f7ea91214f7698bf62e3c85274ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } } }
I20260926 19:36:47.909619 22558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:36:47.911639 22566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:36:47.916980 22566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:36:47.926699 22566 catalog_manager.cc:1347] Generated new cluster ID: bc8141f4e81b4fe9a756a120498284c9
I20260926 19:36:47.926851 22566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:36:47.936364 22566 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:36:47.938344 22566 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:36:47.948006 22566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Generated new TSK 0
I20260926 19:36:47.949198 22566 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:36:48.012727 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42151
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:48.363845 22578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:48.364144 22578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:48.364239 22578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:48.371964 22578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:48.372090 22578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:48.372205 22578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:36:48.382560 22578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:48.383854 22578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:48.385192 22578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:48.399405 22583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:48.399449 22584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:48.399627 22586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:48.400552 22578 server_base.cc:1034] running on GCE node
I20260926 19:36:48.401310 22578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:48.402407 22578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:48.403642 22578 hybrid_clock.cc:648] HybridClock initialized: now 1790451408403596 us; error 40 us; skew 500 ppm
I20260926 19:36:48.403996 22578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:48 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:48.407405 22578 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:36:48.408941 22578 webserver.cc:460] Webserver started at http://127.6.215.193:38015/ using document root <none> and password file <none>
I20260926 19:36:48.409682 22578 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:48.409870 22578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:48.410202 22578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:48.413260 22578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "f7071382e39f4bf1a300880c1e682f01"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "0d6293577491651b569c627aa37122a0"
server_key_iv: "b782812b3500499ab94754aebb13e552"
server_key_version: "encryptionkey@0"
I20260926 19:36:48.414058 22578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "f7071382e39f4bf1a300880c1e682f01"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "0d6293577491651b569c627aa37122a0"
server_key_iv: "b782812b3500499ab94754aebb13e552"
server_key_version: "encryptionkey@0"
I20260926 19:36:48.419518 22578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260926 19:36:48.423252 22593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:48.424593 22578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 19:36:48.424827 22578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "f7071382e39f4bf1a300880c1e682f01"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "0d6293577491651b569c627aa37122a0"
server_key_iv: "b782812b3500499ab94754aebb13e552"
server_key_version: "encryptionkey@0"
I20260926 19:36:48.425051 22578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:48.448213 22578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:48.452656 22578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:48.452981 22578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:48.453989 22578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:48.455325 22578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:48.455431 22578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:48.455583 22578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:48.455657 22578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:48.489686 22578 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:38817
I20260926 19:36:48.489776 22705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:38817 every 8 connection(s)
I20260926 19:36:48.490965 22578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:36:48.498739 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 22578
I20260926 19:36:48.498967 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:36:48.499632 7007 external_mini_cluster.cc:1428] Setting key 2748b97d5ebb4f317cb64850895b088a
Sep 26 19:36:48 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:48.513579 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:48.493729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:52491 (local address 127.6.215.254:42151)
0926 19:36:48.494165 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:48.494170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:48.495206 (+ 1036us) server_negotiation.cc:407] Connection header received
0926 19:36:48.497102 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:48.497109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:48.497202 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:48.497324 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:48.498823 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:48.499663 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:48.501878 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:48.502337 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:48.502497 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:48.506653 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:48.506658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:48.506661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:48.506693 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:48.509791 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:48.510774 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:48.510783 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:48.510790 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:48.510943 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:48.511493 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:48.511500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:48.511507 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:48.511878 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:48.512041 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:48.512755 (+ 714us) server_negotiation.cc:299] Negotiation successful
0926 19:36:48.513084 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":228,"threads_started":1}
I20260926 19:36:48.514240 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:48.494236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:52491)
0926 19:36:48.495053 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:48.495089 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:48.496829 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:48.497569 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:48.497578 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:48.498090 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:48.498646 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:48.498662 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:48.499887 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:48.499918 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:48.501730 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:48.501746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:48.502647 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:48.502655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:48.502919 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:48.503664 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:48.503687 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:48.506398 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:48.509977 (+ 3579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:48.509986 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:48.510008 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:48.510636 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:48.511123 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:48.511127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:48.511130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:48.511368 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:48.512004 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:48.512010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:48.512345 (+ 335us) client_negotiation.cc:769] Sending connection context
0926 19:36:48.512583 (+ 238us) client_negotiation.cc:240] Negotiation successful
0926 19:36:48.513056 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":349,"threads_started":1}
I20260926 19:36:48.516024 22706 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42151
I20260926 19:36:48.516366 22706 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:48.517177 22706 heartbeater.cc:511] Master 127.6.215.254:42151 requested a full tablet report, sending...
I20260926 19:36:48.519424 22517 ts_manager.cc:194] Registered new tserver with Master: f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817)
I20260926 19:36:48.521139 22517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:52491
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:36:48.557294 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42151
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:48.879580 22712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:48.879904 22712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:48.880066 22712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:48.887269 22712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:48.887441 22712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:48.887636 22712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:36:48.897324 22712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:48.898607 22712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:48.899858 22712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:48.912614 22717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:48.912693 22720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:48.912658 22718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:48.913705 22712 server_base.cc:1034] running on GCE node
I20260926 19:36:48.914222 22712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:48.915060 22712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:48.916285 22712 hybrid_clock.cc:648] HybridClock initialized: now 1790451408916234 us; error 40 us; skew 500 ppm
I20260926 19:36:48.916555 22712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:48 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:48.919715 22712 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:36:48.921252 22712 webserver.cc:460] Webserver started at http://127.6.215.194:34885/ using document root <none> and password file <none>
I20260926 19:36:48.921917 22712 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:48.922096 22712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:48.922451 22712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:48.925315 22712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "88e69e83e6ca46279a0cd10b2395390a"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "ff2399e8598e424126642adaf396e830"
server_key_iv: "7026d07a22866bc3063d3244e70c4372"
server_key_version: "encryptionkey@0"
I20260926 19:36:48.925983 22712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "88e69e83e6ca46279a0cd10b2395390a"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "ff2399e8598e424126642adaf396e830"
server_key_iv: "7026d07a22866bc3063d3244e70c4372"
server_key_version: "encryptionkey@0"
I20260926 19:36:48.931137 22712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260926 19:36:48.934413 22727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:48.935719 22712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:36:48.935844 22712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "88e69e83e6ca46279a0cd10b2395390a"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "ff2399e8598e424126642adaf396e830"
server_key_iv: "7026d07a22866bc3063d3244e70c4372"
server_key_version: "encryptionkey@0"
I20260926 19:36:48.936007 22712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:48.949469 22712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:48.953505 22712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:48.953752 22712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:48.954640 22712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:48.955909 22712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:48.955979 22712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:48.956059 22712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:48.956110 22712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:48.989754 22712 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:44077
I20260926 19:36:48.989794 22839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:44077 every 8 connection(s)
I20260926 19:36:48.991101 22712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:36:48.999456 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 22712
I20260926 19:36:48.999678 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:36:49.000226 7007 external_mini_cluster.cc:1428] Setting key d509b3c273a4686b0c4e00f0d9bcc21a
Sep 26 19:36:49 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451408, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:49.012228 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:48.993816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:52637 (local address 127.6.215.254:42151)
0926 19:36:48.994063 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:48.994068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:48.995146 (+ 1078us) server_negotiation.cc:407] Connection header received
0926 19:36:48.997031 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:48.997035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:48.997108 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:48.997199 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:48.998619 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:48.999344 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.001441 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.001982 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.002087 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.006242 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:49.006248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:49.006251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:49.006284 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:49.009022 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.009768 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.009773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.009776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.009898 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.010411 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.010420 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.010422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.010737 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:49.010890 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.011652 (+ 762us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.011877 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:36:49.013374 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:48.994260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:52637)
0926 19:36:48.994958 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:48.994999 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:48.996705 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:48.997399 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:48.997407 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:48.997887 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:48.998464 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:48.998482 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:48.999511 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:48.999516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.001281 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.001293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.002207 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.002218 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.002471 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.003245 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:49.003279 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:49.006001 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:49.009175 (+ 3174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:49.009185 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:49.009203 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:49.009629 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:49.010022 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:49.010027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:49.010030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:49.010290 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:49.010864 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:49.010873 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:49.011223 (+ 350us) client_negotiation.cc:769] Sending connection context
0926 19:36:49.011561 (+ 338us) client_negotiation.cc:240] Negotiation successful
0926 19:36:49.012092 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":284,"threads_started":1}
I20260926 19:36:49.014907 22840 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42151
I20260926 19:36:49.015239 22840 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:49.015966 22840 heartbeater.cc:511] Master 127.6.215.254:42151 requested a full tablet report, sending...
I20260926 19:36:49.017586 22517 ts_manager.cc:194] Registered new tserver with Master: 88e69e83e6ca46279a0cd10b2395390a (127.6.215.194:44077)
I20260926 19:36:49.018869 22517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:52637
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:36:49.053925 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42151
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:36:49.379338 22845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:36:49.379632 22845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:36:49.379727 22845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:36:49.387101 22845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:36:49.387224 22845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:36:49.387343 22845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:36:49.397681 22845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:36:49.398916 22845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:36:49.400167 22845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:36:49.415242 22850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:49.415509 22851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:36:49.416400 22853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:36:49.416827 22845 server_base.cc:1034] running on GCE node
I20260926 19:36:49.417616 22845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:36:49.418634 22845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:36:49.419883 22845 hybrid_clock.cc:648] HybridClock initialized: now 1790451409419844 us; error 38 us; skew 500 ppm
I20260926 19:36:49.420214 22845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:36:49 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451409, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:36:49.423420 22845 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:36:49.425022 22845 webserver.cc:460] Webserver started at http://127.6.215.195:36703/ using document root <none> and password file <none>
I20260926 19:36:49.425715 22845 fs_manager.cc:359] Metadata directory not provided
I20260926 19:36:49.425863 22845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:36:49.426183 22845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:36:49.429323 22845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "769c63f067aa4edc813dfa5a461a1235"
format_stamp: "Formatted at 2026-09-26 19:36:49 on dist-test-slave-phbh"
server_key: "e95be1662df65f49e87eaf61d9b93e39"
server_key_iv: "9f421e53f765b414f30559526dc0a1ef"
server_key_version: "encryptionkey@0"
I20260926 19:36:49.430122 22845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "769c63f067aa4edc813dfa5a461a1235"
format_stamp: "Formatted at 2026-09-26 19:36:49 on dist-test-slave-phbh"
server_key: "e95be1662df65f49e87eaf61d9b93e39"
server_key_iv: "9f421e53f765b414f30559526dc0a1ef"
server_key_version: "encryptionkey@0"
I20260926 19:36:49.435405 22845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 19:36:49.439061 22860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:49.440487 22845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:36:49.440686 22845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "769c63f067aa4edc813dfa5a461a1235"
format_stamp: "Formatted at 2026-09-26 19:36:49 on dist-test-slave-phbh"
server_key: "e95be1662df65f49e87eaf61d9b93e39"
server_key_iv: "9f421e53f765b414f30559526dc0a1ef"
server_key_version: "encryptionkey@0"
I20260926 19:36:49.440953 22845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:36:49.462175 22845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:36:49.467053 22845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:36:49.467303 22845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:36:49.468207 22845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:36:49.469833 22845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:36:49.469923 22845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:49.470018 22845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:36:49.470351 22845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:36:49.505716 22845 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:34227
I20260926 19:36:49.505806 22972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:34227 every 8 connection(s)
I20260926 19:36:49.507026 22845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:36:49.507843 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 22845
I20260926 19:36:49.508052 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:36:49.508632 7007 external_mini_cluster.cc:1428] Setting key c371cb4c07dc7563c254854bf3931413
Sep 26 19:36:49 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451409, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:36:49.524122 22706 heartbeater.cc:503] Master 127.6.215.254:42151 was elected leader, sending a full tablet report...
I20260926 19:36:49.529166 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.509828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:57403 (local address 127.6.215.254:42151)
0926 19:36:49.510008 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.510014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.511491 (+ 1477us) server_negotiation.cc:407] Connection header received
0926 19:36:49.513537 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.513554 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.513656 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.513750 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:49.515476 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.516260 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.517991 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.518447 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.518601 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.522125 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:49.522130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:49.522134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:49.522167 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:49.525915 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.526665 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.526671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.526675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.526814 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.527255 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.527260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.527263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.527559 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:49.527722 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.528532 (+ 810us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.528764 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 19:36:49.530098 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.510404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:57403)
0926 19:36:49.511316 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:49.511371 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:49.513263 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:49.514156 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:49.514175 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:49.514705 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:36:49.515275 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.515292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.516414 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.516418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.517845 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.517857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.518720 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.518726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.518943 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.519612 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:36:49.519639 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:36:49.521929 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:36:49.526069 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:49.526079 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:49.526101 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:49.526531 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:49.526931 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:36:49.526936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:36:49.526940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:36:49.527131 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:36:49.527711 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:36:49.527720 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:36:49.528064 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 19:36:49.528411 (+ 347us) client_negotiation.cc:240] Negotiation successful
0926 19:36:49.528985 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":309,"threads_started":1}
I20260926 19:36:49.531458 22973 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42151
I20260926 19:36:49.531810 22973 heartbeater.cc:464] Registering TS with master...
I20260926 19:36:49.532558 22973 heartbeater.cc:511] Master 127.6.215.254:42151 requested a full tablet report, sending...
I20260926 19:36:49.534000 22517 ts_manager.cc:194] Registered new tserver with Master: 769c63f067aa4edc813dfa5a461a1235 (127.6.215.195:34227)
I20260926 19:36:49.535128 22517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:57403
I20260926 19:36:49.537365 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:36:49.555444 22708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.542123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.6.215.254:42151)
0926 19:36:49.542287 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.542292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.542749 (+ 457us) server_negotiation.cc:407] Connection header received
0926 19:36:49.542884 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.542888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.542949 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.543054 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:49.544517 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.545446 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.546857 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.547302 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.547406 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.550236 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:49.550240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:49.550244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:49.550277 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:49.552953 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.553726 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.553730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.553733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.553863 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.554222 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.554225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.554226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.554537 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:49.554771 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.555023 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.555189 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:36:49.564080 22517 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:36:49.571035 22517 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:36:49.596207 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.583717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.6.215.195:34227)
0926 19:36:49.585143 (+ 1426us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.585153 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.586315 (+ 1162us) server_negotiation.cc:407] Connection header received
0926 19:36:49.586657 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.586665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.586937 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.587114 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:49.588185 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.589895 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.593149 (+ 3254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.594615 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.594712 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.594958 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.595649 (+ 691us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.595820 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1263,"thread_start_us":1102,"threads_started":1}
I20260926 19:36:49.597380 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.583382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34227 (local address 127.0.0.1:36262)
0926 19:36:49.586165 (+ 2783us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:49.586213 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:49.586392 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:49.587488 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:49.587495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:49.587528 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:49.587951 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.587966 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.590080 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.590098 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.592961 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.592976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.595157 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.595164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.595357 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.595419 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 19:36:49.596730 (+ 1311us) client_negotiation.cc:240] Negotiation successful
0926 19:36:49.596976 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2522,"thread_start_us":2341,"threads_started":1}
I20260926 19:36:49.597771 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.583847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:44077 (local address 127.0.0.1:43770)
0926 19:36:49.586165 (+ 2318us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:49.586213 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:49.586392 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:49.588425 (+ 2033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:49.588431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:49.588454 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:49.588885 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.588898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.591809 (+ 2911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.591814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.593936 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.593951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.596995 (+ 3044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.597001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.597192 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.597224 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 19:36:49.597298 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 19:36:49.597443 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2116,"thread_start_us":1872,"threads_started":1}
I20260926 19:36:49.598604 22988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.584517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.6.215.194:44077)
0926 19:36:49.585541 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.585550 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.586424 (+ 874us) server_negotiation.cc:407] Connection header received
0926 19:36:49.587754 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.587762 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.587994 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.588181 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:49.589083 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.591644 (+ 2561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.594119 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.596562 (+ 2443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.596718 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.598156 (+ 1438us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.598234 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.598346 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":762,"threads_started":1}
I20260926 19:36:49.602514 22908 tablet_service.cc:1467] Processing CreateTablet for tablet 9075865652684dadb43e792515d81289 (DEFAULT_TABLE table=test-table [id=a095e3ab395740c885ec9bcdfb5d15e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:49.603801 22908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9075865652684dadb43e792515d81289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:49.604226 22775 tablet_service.cc:1467] Processing CreateTablet for tablet 9075865652684dadb43e792515d81289 (DEFAULT_TABLE table=test-table [id=a095e3ab395740c885ec9bcdfb5d15e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:49.605527 22775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9075865652684dadb43e792515d81289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:49.608496 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.586555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:38817 (local address 127.0.0.1:33196)
0926 19:36:49.597833 (+ 11278us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:49.597857 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:49.597954 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:49.601794 (+ 3840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:49.601799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:49.601826 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:49.601992 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.602003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.603984 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.603989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.605867 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.605883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.607702 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.607709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.607898 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.607933 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:36:49.608050 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 19:36:49.608201 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11142,"thread_start_us":13977,"threads_started":1}
I20260926 19:36:49.608621 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.586838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33196 (local address 127.6.215.193:38817)
0926 19:36:49.601134 (+ 14296us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.601141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.601165 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:36:49.601237 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.601242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.601556 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.601793 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:49.602134 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.603810 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.606057 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.607506 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.607695 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.607910 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.608103 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.608285 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14138,"thread_start_us":13972,"threads_started":1}
I20260926 19:36:49.612380 22641 tablet_service.cc:1467] Processing CreateTablet for tablet 9075865652684dadb43e792515d81289 (DEFAULT_TABLE table=test-table [id=a095e3ab395740c885ec9bcdfb5d15e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:36:49.613917 22641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9075865652684dadb43e792515d81289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:36:49.616508 22991 tablet_bootstrap.cc:492] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Bootstrap starting.
I20260926 19:36:49.619773 22992 tablet_bootstrap.cc:492] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Bootstrap starting.
I20260926 19:36:49.619961 22991 tablet_bootstrap.cc:654] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:49.621328 22991 log.cc:824] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:49.623847 22992 tablet_bootstrap.cc:654] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:49.625412 22992 log.cc:824] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:49.627171 22995 tablet_bootstrap.cc:492] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Bootstrap starting.
I20260926 19:36:49.629283 22992 tablet_bootstrap.cc:492] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: No bootstrap required, opened a new log
I20260926 19:36:49.629573 22992 ts_tablet_manager.cc:1397] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 19:36:49.630416 22991 tablet_bootstrap.cc:492] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: No bootstrap required, opened a new log
I20260926 19:36:49.630702 22991 ts_tablet_manager.cc:1397] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260926 19:36:49.630800 22995 tablet_bootstrap.cc:654] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Neither blocks nor log segments found. Creating new log.
I20260926 19:36:49.631999 22995 log.cc:824] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Log is configured to *not* fsync() on all Append() calls
I20260926 19:36:49.637753 22995 tablet_bootstrap.cc:492] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: No bootstrap required, opened a new log
I20260926 19:36:49.638019 22995 ts_tablet_manager.cc:1397] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260926 19:36:49.639263 22991 raft_consensus.cc:348] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.639743 22991 raft_consensus.cc:374] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:49.639868 22991 raft_consensus.cc:729] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769c63f067aa4edc813dfa5a461a1235, State: Initialized, Role: FOLLOWER
I20260926 19:36:49.640553 22991 consensus_queue.cc:260] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.640659 22992 raft_consensus.cc:348] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.641167 22992 raft_consensus.cc:374] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:49.641302 22992 raft_consensus.cc:729] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e69e83e6ca46279a0cd10b2395390a, State: Initialized, Role: FOLLOWER
I20260926 19:36:49.641752 22973 heartbeater.cc:503] Master 127.6.215.254:42151 was elected leader, sending a full tablet report...
I20260926 19:36:49.642259 22991 ts_tablet_manager.cc:1428] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260926 19:36:49.643447 22992 consensus_queue.cc:260] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.644642 22840 heartbeater.cc:503] Master 127.6.215.254:42151 was elected leader, sending a full tablet report...
I20260926 19:36:49.645273 22992 ts_tablet_manager.cc:1428] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260926 19:36:49.648264 22995 raft_consensus.cc:348] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.648712 22995 raft_consensus.cc:374] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:36:49.648834 22995 raft_consensus.cc:729] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7071382e39f4bf1a300880c1e682f01, State: Initialized, Role: FOLLOWER
I20260926 19:36:49.649433 22995 consensus_queue.cc:260] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.650785 22995 ts_tablet_manager.cc:1428] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
W20260926 19:36:49.743129 22841 tablet.cc:2369] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:36:49.743345 22707 tablet.cc:2369] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:36:49.758802 22974 tablet.cc:2369] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:36:49.842410 22997 raft_consensus.cc:482] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:36:49.842761 22997 raft_consensus.cc:504] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.844558 22997 leader_election.cc:288] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88e69e83e6ca46279a0cd10b2395390a (127.6.215.194:44077), f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817)
I20260926 19:36:49.850693 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.845024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:44077 (local address 127.6.215.195:55403)
0926 19:36:49.845228 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:49.845244 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:49.845341 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:49.845735 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:49.845742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:49.845772 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:49.845987 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.845998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.847269 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.847274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.848937 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.848951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.850134 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.850139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.850288 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.850314 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 19:36:49.850399 (+ 85us) client_negotiation.cc:240] Negotiation successful
0926 19:36:49.850491 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260926 19:36:49.850828 22988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.845075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:55403 (local address 127.6.215.194:44077)
0926 19:36:49.845285 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.845292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.845313 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:36:49.845489 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.845495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.845592 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.845703 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:49.846162 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.847096 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.849127 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.849964 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.850140 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.850217 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.850444 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.850575 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:36:49.852038 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.845784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:47857 (local address 127.6.215.193:38817)
0926 19:36:49.845990 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.845996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.846431 (+ 435us) server_negotiation.cc:407] Connection header received
0926 19:36:49.846551 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.846558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.846642 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.846733 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:36:49.847228 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.847934 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.849933 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.851155 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.851324 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.851402 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.851782 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.851874 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 19:36:49.851907 22795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9075865652684dadb43e792515d81289" candidate_uuid: "769c63f067aa4edc813dfa5a461a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e69e83e6ca46279a0cd10b2395390a" is_pre_election: true
I20260926 19:36:49.852206 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.845500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:38817 (local address 127.6.215.195:47857)
0926 19:36:49.846323 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0926 19:36:49.846337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:36:49.846435 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:36:49.846767 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:36:49.846770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:36:49.846789 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:36:49.847077 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.847086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.848092 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.848097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.849765 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:36:49.849776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.851336 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.851365 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:36:49.851609 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.851644 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 19:36:49.851740 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 19:36:49.851863 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":520,"threads_started":1}
I20260926 19:36:49.852506 22795 raft_consensus.cc:2454] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769c63f067aa4edc813dfa5a461a1235 in term 0.
I20260926 19:36:49.853266 22862 leader_election.cc:302] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 769c63f067aa4edc813dfa5a461a1235, 88e69e83e6ca46279a0cd10b2395390a; no voters:
I20260926 19:36:49.853262 22661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9075865652684dadb43e792515d81289" candidate_uuid: "769c63f067aa4edc813dfa5a461a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7071382e39f4bf1a300880c1e682f01" is_pre_election: true
I20260926 19:36:49.853685 22997 raft_consensus.cc:2781] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:36:49.853847 22997 raft_consensus.cc:482] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:36:49.853811 22661 raft_consensus.cc:2454] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 769c63f067aa4edc813dfa5a461a1235 in term 0.
I20260926 19:36:49.853956 22997 raft_consensus.cc:3037] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:49.856134 22997 raft_consensus.cc:504] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.856832 22997 leader_election.cc:288] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [CANDIDATE]: Term 1 election: Requested vote from peers 88e69e83e6ca46279a0cd10b2395390a (127.6.215.194:44077), f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817)
I20260926 19:36:49.857295 22795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9075865652684dadb43e792515d81289" candidate_uuid: "769c63f067aa4edc813dfa5a461a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e69e83e6ca46279a0cd10b2395390a"
I20260926 19:36:49.857415 22661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9075865652684dadb43e792515d81289" candidate_uuid: "769c63f067aa4edc813dfa5a461a1235" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7071382e39f4bf1a300880c1e682f01"
I20260926 19:36:49.857513 22795 raft_consensus.cc:3037] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:49.857659 22661 raft_consensus.cc:3037] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:36:49.860793 22661 raft_consensus.cc:2454] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769c63f067aa4edc813dfa5a461a1235 in term 1.
I20260926 19:36:49.860795 22795 raft_consensus.cc:2454] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 769c63f067aa4edc813dfa5a461a1235 in term 1.
I20260926 19:36:49.861202 22862 leader_election.cc:302] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 769c63f067aa4edc813dfa5a461a1235, 88e69e83e6ca46279a0cd10b2395390a; no voters:
I20260926 19:36:49.861497 22997 raft_consensus.cc:2781] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:36:49.861928 22997 raft_consensus.cc:686] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 1 LEADER]: Becoming Leader. State: Replica: 769c63f067aa4edc813dfa5a461a1235, State: Running, Role: LEADER
I20260926 19:36:49.862453 22997 consensus_queue.cc:237] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } }
I20260926 19:36:49.867554 22516 catalog_manager.cc:5495] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 reported cstate change: term changed from 0 to 1, leader changed from <none> to 769c63f067aa4edc813dfa5a461a1235 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "769c63f067aa4edc813dfa5a461a1235" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } health_report { overall_health: UNKNOWN } } }
I20260926 19:36:49.889864 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.884309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.6.215.195:34227)
0926 19:36:49.884511 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.884517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.884615 (+ 98us) server_negotiation.cc:407] Connection header received
0926 19:36:49.884941 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.884946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.885031 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.885185 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:36:49.885578 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.886294 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.887687 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.888092 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.888233 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.888747 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:36:49.889285 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:36:49.889441 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.889520 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.889602 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 19:36:49.897838 22661 raft_consensus.cc:1261] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Refusing update from remote peer 769c63f067aa4edc813dfa5a461a1235: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:49.897917 22795 raft_consensus.cc:1261] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 1 FOLLOWER]: Refusing update from remote peer 769c63f067aa4edc813dfa5a461a1235: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:36:49.898789 23001 consensus_queue.cc:1035] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:49.899230 22997 consensus_queue.cc:1035] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:36:49.913738 23006 mvcc.cc:205] Tried to move back new op lower bound from 7333688974930870272 to 7333688974800699392. Current Snapshot: MvccSnapshot[applied={T|T < 7333688974930870272}]
I20260926 19:36:49.926651 22516 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:49.926937 22516 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:36:49.930907 22516 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Sending DeleteTablet for 3 replicas of tablet 9075865652684dadb43e792515d81289
I20260926 19:36:49.932212 22908 tablet_service.cc:1514] Processing DeleteTablet for tablet 9075865652684dadb43e792515d81289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:49 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:36262
I20260926 19:36:49.932204 22775 tablet_service.cc:1514] Processing DeleteTablet for tablet 9075865652684dadb43e792515d81289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:49 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:43770
I20260926 19:36:49.932322 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 22578
I20260926 19:36:49.933205 22641 tablet_service.cc:1514] Processing DeleteTablet for tablet 9075865652684dadb43e792515d81289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:36:49 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33196
I20260926 19:36:49.944201 23014 tablet_replica.cc:330] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: stopping tablet replica
I20260926 19:36:49.945420 23014 raft_consensus.cc:2229] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:36:49.946921 23014 raft_consensus.cc:2258] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:49.947817 23017 tablet_replica.cc:330] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: stopping tablet replica
I20260926 19:36:49.948303 23017 raft_consensus.cc:2229] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:49.949333 23015 tablet_replica.cc:330] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: stopping tablet replica
I20260926 19:36:49.949429 23017 raft_consensus.cc:2258] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:49.949815 23015 raft_consensus.cc:2229] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:36:49.950349 23015 raft_consensus.cc:2258] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:36:49.952179 23014 ts_tablet_manager.cc:1905] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:49.952349 23015 ts_tablet_manager.cc:1905] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:36:49.953729 23017 ts_tablet_manager.cc:1905] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 19:36:49 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:36:49.959416 23015 ts_tablet_manager.cc:1918] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:49.959631 23015 log.cc:1196] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/9075865652684dadb43e792515d81289
I20260926 19:36:49.960189 23015 ts_tablet_manager.cc:1939] T 9075865652684dadb43e792515d81289 P 88e69e83e6ca46279a0cd10b2395390a: Deleting consensus metadata
I20260926 19:36:49.960937 23017 ts_tablet_manager.cc:1918] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:49.961172 23017 log.cc:1196] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/9075865652684dadb43e792515d81289
I20260926 19:36:49.961738 23017 ts_tablet_manager.cc:1939] T 9075865652684dadb43e792515d81289 P f7071382e39f4bf1a300880c1e682f01: Deleting consensus metadata
I20260926 19:36:49.962311 22502 catalog_manager.cc:4841] TS 88e69e83e6ca46279a0cd10b2395390a (127.6.215.194:44077): tablet 9075865652684dadb43e792515d81289 (table test-table [id=a095e3ab395740c885ec9bcdfb5d15e8]) successfully deleted
I20260926 19:36:49.963501 22502 catalog_manager.cc:4841] TS f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817): tablet 9075865652684dadb43e792515d81289 (table test-table [id=a095e3ab395740c885ec9bcdfb5d15e8]) successfully deleted
I20260926 19:36:49.964521 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:49.933601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.6.215.193:38817)
0926 19:36:49.933818 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:49.933827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:49.941794 (+ 7967us) server_negotiation.cc:407] Connection header received
0926 19:36:49.942007 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:49.942015 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:49.942147 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:49.942262 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:49.946368 (+ 4106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.947679 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.949835 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:49.950473 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:49.950587 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:49.956217 (+ 5630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:49.956232 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:49.956248 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:49.956304 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:49.960674 (+ 4370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.961817 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.961828 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.961831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.962002 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:49.962480 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:49.962486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:49.962489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:49.962957 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:49.963286 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:49.963931 (+ 645us) server_negotiation.cc:299] Negotiation successful
0926 19:36:49.964168 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:36:49.965175 22701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:49.973868 23014 ts_tablet_manager.cc:1918] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:36:49.974102 23014 log.cc:1196] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/9075865652684dadb43e792515d81289
I20260926 19:36:49.974706 23014 ts_tablet_manager.cc:1939] T 9075865652684dadb43e792515d81289 P 769c63f067aa4edc813dfa5a461a1235: Deleting consensus metadata
I20260926 19:36:49.976807 22503 catalog_manager.cc:4841] TS 769c63f067aa4edc813dfa5a461a1235 (127.6.215.195:34227): tablet 9075865652684dadb43e792515d81289 (table test-table [id=a095e3ab395740c885ec9bcdfb5d15e8]) successfully deleted
I20260926 19:36:50.060835 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 22578
I20260926 19:36:50.085152 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 22712
Sep 26 19:36:50 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:36:50.103051 22988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:50.085938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.6.215.194:44077)
0926 19:36:50.086101 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:50.086106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:50.086123 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:36:50.086312 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:50.086316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:50.086380 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:50.086470 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:50.088154 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:50.089067 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:50.091013 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:50.091370 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:50.091459 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:50.096882 (+ 5423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:50.096898 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:50.096914 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:50.096953 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:50.099950 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:50.100825 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:50.100842 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:50.100846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:50.100966 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:50.101396 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:50.101399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:50.101401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:50.101843 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:50.102144 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:50.102447 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 19:36:50.102678 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 19:36:50.103780 22835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:50.192656 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 22712
I20260926 19:36:50.210768 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 22845
Sep 26 19:36:50 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:36:50.227504 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:36:50.211601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.6.215.195:34227)
0926 19:36:50.211766 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 19:36:50.211771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:36:50.211950 (+ 179us) server_negotiation.cc:407] Connection header received
0926 19:36:50.212160 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:36:50.212166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:36:50.212271 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:36:50.212385 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:36:50.213754 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:50.214573 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:50.215971 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:36:50.216447 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:36:50.216592 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:36:50.221170 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:36:50.221186 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:36:50.221200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:36:50.221240 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:36:50.224426 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:50.225313 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:50.225329 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:50.225332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:50.225488 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:36:50.225914 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:36:50.225917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:36:50.225919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:36:50.226256 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:36:50.226581 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0926 19:36:50.226904 (+ 323us) server_negotiation.cc:299] Negotiation successful
0926 19:36:50.227128 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 19:36:50.228193 22968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:50.350092 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 22845
I20260926 19:36:50.369544 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 22485
I20260926 19:36:50.370749 22547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:36:50.486675 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 22485
I20260926 19:37:10.509033 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42151
--webserver_interface=127.6.215.254
--webserver_port=44071
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:10.836364 23022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:10.836649 23022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:10.836822 23022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:10.844105 23022 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 19:37:10.844200 23022 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 19:37:10.844280 23022 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 19:37:10.844380 23022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:37:10.844452 23022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:10.844574 23022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:10.844647 23022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:37:10.844725 23022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:37:10.854794 23022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:10.856177 23022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:10.857547 23022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:10.879773 23029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:10.879809 23031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:10.879773 23028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:10.880856 23022 server_base.cc:1034] running on GCE node
I20260926 19:37:10.881845 23022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:10.883421 23022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:10.884645 23022 hybrid_clock.cc:648] HybridClock initialized: now 1790451430884583 us; error 32 us; skew 500 ppm
I20260926 19:37:10.885064 23022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:10 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451430, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:10.888522 23022 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:37:10.890148 23022 webserver.cc:460] Webserver started at http://127.6.215.254:44071/ using document root <none> and password file <none>
I20260926 19:37:10.890862 23022 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:10.890961 23022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:10.896755 23022 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 19:37:10.899508 23038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:10.900872 23022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 19:37:10.901041 23022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2107f7ea91214f7698bf62e3c85274ff"
format_stamp: "Formatted at 2026-09-26 19:36:47 on dist-test-slave-phbh"
server_key: "7663e621ce5e2930b66a22ed4918d18e"
server_key_iv: "a54a72a46fe8be5db52918c4b5eb1528"
server_key_version: "encryptionkey@0"
I20260926 19:37:10.902119 23022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:10.919178 23022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:10.923319 23022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:10.923632 23022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:10.942610 23022 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42151
I20260926 19:37:10.942641 23089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42151 every 8 connection(s)
I20260926 19:37:10.944186 23022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:37:10.951249 23090 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:37:10.952744 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23022
I20260926 19:37:10.953823 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:38817
--local_ip_for_outbound_sockets=127.6.215.193
--tserver_master_addrs=127.6.215.254:42151
--webserver_port=38015
--webserver_interface=127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:37:10.956017 23090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Bootstrap starting.
I20260926 19:37:10.977598 23090 log.cc:824] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:10.989662 23090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:37:10.990237 23090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Bootstrap complete.
I20260926 19:37:10.999323 23090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:11.000212 23090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2107f7ea91214f7698bf62e3c85274ff, State: Initialized, Role: FOLLOWER
I20260926 19:37:11.000905 23090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:11.001151 23090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:37:11.001276 23090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:37:11.001463 23090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 1 FOLLOWER]: Advancing to term 2
I20260926 19:37:11.005458 23090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:11.005992 23090 leader_election.cc:302] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2107f7ea91214f7698bf62e3c85274ff; no voters:
I20260926 19:37:11.006635 23090 leader_election.cc:288] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 19:37:11.006755 23094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 FOLLOWER]: Leader election won for term 2
I20260926 19:37:11.009176 23090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:37:11.013852 23094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 LEADER]: Becoming Leader. State: Replica: 2107f7ea91214f7698bf62e3c85274ff, State: Running, Role: LEADER
I20260926 19:37:11.014644 23094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:11.017514 23095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2107f7ea91214f7698bf62e3c85274ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } } }
I20260926 19:37:11.017956 23095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:11.018543 23096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2107f7ea91214f7698bf62e3c85274ff. Latest consensus state: current_term: 2 leader_uuid: "2107f7ea91214f7698bf62e3c85274ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } } }
I20260926 19:37:11.018852 23096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:11.020898 23109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:37:11.029449 23109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:37:11.030849 23109 catalog_manager.cc:1259] Loaded cluster ID: bc8141f4e81b4fe9a756a120498284c9
I20260926 19:37:11.030987 23109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:37:11.035033 23109 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:37:11.036545 23109 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Loaded TSK: 0
I20260926 19:37:11.071039 23109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Generated new TSK 1
I20260926 19:37:11.071465 23109 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 19:37:11.293090 23092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:11.293355 23092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:11.293507 23092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:11.300660 23092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:11.300787 23092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:11.300992 23092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:37:11.311187 23092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:38817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=38015
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:11.312594 23092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:11.314057 23092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:11.326642 23119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:11.326792 23121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:11.327387 23118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:11.328598 23092 server_base.cc:1034] running on GCE node
I20260926 19:37:11.329381 23092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:11.330582 23092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:11.331841 23092 hybrid_clock.cc:648] HybridClock initialized: now 1790451431331794 us; error 57 us; skew 500 ppm
I20260926 19:37:11.332252 23092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:11 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:11.336282 23092 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:37:11.338097 23092 webserver.cc:460] Webserver started at http://127.6.215.193:38015/ using document root <none> and password file <none>
I20260926 19:37:11.339143 23092 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:11.339316 23092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:11.346457 23092 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260926 19:37:11.350006 23128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:11.351658 23092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:37:11.351841 23092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "f7071382e39f4bf1a300880c1e682f01"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "0d6293577491651b569c627aa37122a0"
server_key_iv: "b782812b3500499ab94754aebb13e552"
server_key_version: "encryptionkey@0"
I20260926 19:37:11.353185 23092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:11.380136 23092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:11.385386 23092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:11.385659 23092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:11.386799 23092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:11.388397 23092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:11.388500 23092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:11.388607 23092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:11.388676 23092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:11.430733 23092 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:38817
I20260926 19:37:11.430827 23240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:38817 every 8 connection(s)
I20260926 19:37:11.432243 23092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:37:11.440714 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23092
I20260926 19:37:11.441358 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:44077
--local_ip_for_outbound_sockets=127.6.215.194
--tserver_master_addrs=127.6.215.254:42151
--webserver_port=34885
--webserver_interface=127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:37:11 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:11.458392 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:11.435284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:35221)
0926 19:37:11.436121 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:11.436166 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:11.437953 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:11.439322 (+ 1369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:11.439330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:11.439908 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:11.440688 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:11.440714 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.442442 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.442450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:11.444532 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:11.444549 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.445577 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.445586 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:11.445909 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:11.446664 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:37:11.446694 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:11.449412 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:11.454164 (+ 4752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:11.454170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:11.454183 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:11.454610 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:11.455097 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:11.455100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:11.455102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:11.455274 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:11.456229 (+ 955us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:11.456235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:11.456526 (+ 291us) client_negotiation.cc:769] Sending connection context
0926 19:37:11.456753 (+ 227us) client_negotiation.cc:240] Negotiation successful
0926 19:37:11.457214 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":353,"threads_started":1}
I20260926 19:37:11.458722 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:11.434991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:35221 (local address 127.6.215.254:42151)
0926 19:37:11.435520 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:11.435531 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:11.436302 (+ 771us) server_negotiation.cc:407] Connection header received
0926 19:37:11.438501 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:11.438513 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:11.438892 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:11.439168 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:11.440944 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.442269 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.444669 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.445348 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.445475 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:11.449746 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:11.449759 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:11.449782 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:11.449833 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:11.453999 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:11.454766 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:11.454777 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:11.454787 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:11.454991 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:11.455391 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:11.455399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:11.455404 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:11.456093 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:11.456291 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:11.457053 (+ 762us) server_negotiation.cc:299] Negotiation successful
0926 19:37:11.457498 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":232,"threads_started":1}
I20260926 19:37:11.461586 23241 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42151
I20260926 19:37:11.461970 23241 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:11.462920 23241 heartbeater.cc:511] Master 127.6.215.254:42151 requested a full tablet report, sending...
I20260926 19:37:11.465627 23055 ts_manager.cc:194] Registered new tserver with Master: f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817)
I20260926 19:37:11.467136 23055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:35221
W20260926 19:37:11.770532 23245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:11.770809 23245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:11.770901 23245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:11.777980 23245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:11.778090 23245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:11.778213 23245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:37:11.787654 23245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:44077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=34885
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:11.788856 23245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:11.790067 23245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:11.803330 23251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:11.803308 23250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:11.803395 23253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:11.804035 23245 server_base.cc:1034] running on GCE node
I20260926 19:37:11.804837 23245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:11.805794 23245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:11.807036 23245 hybrid_clock.cc:648] HybridClock initialized: now 1790451431806946 us; error 100 us; skew 500 ppm
I20260926 19:37:11.807411 23245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:11 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:11.811555 23245 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:37:11.813234 23245 webserver.cc:460] Webserver started at http://127.6.215.194:34885/ using document root <none> and password file <none>
I20260926 19:37:11.813930 23245 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:11.814071 23245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:11.819480 23245 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.004s
I20260926 19:37:11.822237 23260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:11.823550 23245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:37:11.823725 23245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "88e69e83e6ca46279a0cd10b2395390a"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "ff2399e8598e424126642adaf396e830"
server_key_iv: "7026d07a22866bc3063d3244e70c4372"
server_key_version: "encryptionkey@0"
I20260926 19:37:11.824685 23245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:11.850626 23245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:11.854620 23245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:11.854902 23245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:11.855856 23245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:11.857272 23245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:11.857385 23245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:11.857503 23245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:11.857584 23245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:11.890368 23245 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:44077
I20260926 19:37:11.890453 23372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:44077 every 8 connection(s)
I20260926 19:37:11.891803 23245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:37:11.899770 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23245
I20260926 19:37:11.900555 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:34227
--local_ip_for_outbound_sockets=127.6.215.195
--tserver_master_addrs=127.6.215.254:42151
--webserver_port=36703
--webserver_interface=127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 19:37:11 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:11.913738 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:11.894336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:46473 (local address 127.6.215.254:42151)
0926 19:37:11.894549 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:11.894555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:11.895585 (+ 1030us) server_negotiation.cc:407] Connection header received
0926 19:37:11.897488 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:11.897492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:11.897559 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:11.897646 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:11.899039 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.899734 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.902335 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.902767 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.902861 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:11.907406 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:11.907415 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:11.907423 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:11.907471 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:11.910290 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:11.911288 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:11.911292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:11.911294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:11.911412 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:11.911866 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:11.911869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:11.911870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:11.912196 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:11.912354 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:11.913239 (+ 885us) server_negotiation.cc:299] Negotiation successful
0926 19:37:11.913436 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 19:37:11.915124 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:11.894765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:46473)
0926 19:37:11.895428 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:11.895464 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:11.897238 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:11.897832 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:11.897840 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:11.898382 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:11.898901 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:11.898916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.899892 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.899899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:11.902192 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:11.902210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:11.902989 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:11.902998 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:11.903320 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:11.904078 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:37:11.904113 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:11.907144 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:11.910414 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:11.910425 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:11.910443 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:11.911059 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:11.911516 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:11.911524 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:11.911530 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:11.911760 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:11.912322 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:11.912332 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:11.912829 (+ 497us) client_negotiation.cc:769] Sending connection context
0926 19:37:11.913144 (+ 315us) client_negotiation.cc:240] Negotiation successful
0926 19:37:11.913707 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":269,"threads_started":1}
I20260926 19:37:11.916585 23373 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42151
I20260926 19:37:11.916929 23373 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:11.917623 23373 heartbeater.cc:511] Master 127.6.215.254:42151 requested a full tablet report, sending...
I20260926 19:37:11.919036 23055 ts_manager.cc:194] Registered new tserver with Master: 88e69e83e6ca46279a0cd10b2395390a (127.6.215.194:44077)
I20260926 19:37:11.920310 23055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:46473
W20260926 19:37:12.225118 23376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:12.225394 23376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:12.225487 23376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:12.233008 23376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:12.233120 23376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:12.233240 23376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:37:12.242821 23376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:34227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=36703
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:12.244058 23376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:12.245263 23376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:12.257978 23382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:12.257926 23381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:12.258013 23384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:12.258958 23376 server_base.cc:1034] running on GCE node
I20260926 19:37:12.259544 23376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:12.260457 23376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:12.261667 23376 hybrid_clock.cc:648] HybridClock initialized: now 1790451432261620 us; error 53 us; skew 500 ppm
I20260926 19:37:12.261960 23376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:12 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:12.264962 23376 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:37:12.266289 23376 webserver.cc:460] Webserver started at http://127.6.215.195:36703/ using document root <none> and password file <none>
I20260926 19:37:12.266908 23376 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:12.267023 23376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:12.271992 23376 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:37:12.274499 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:12.275705 23376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 19:37:12.275833 23376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "769c63f067aa4edc813dfa5a461a1235"
format_stamp: "Formatted at 2026-09-26 19:36:49 on dist-test-slave-phbh"
server_key: "e95be1662df65f49e87eaf61d9b93e39"
server_key_iv: "9f421e53f765b414f30559526dc0a1ef"
server_key_version: "encryptionkey@0"
I20260926 19:37:12.276723 23376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:12.323180 23376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:12.327020 23376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:12.327250 23376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:12.328207 23376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:12.329723 23376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:12.329819 23376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:12.329898 23376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:12.329954 23376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:12.362182 23376 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:34227
I20260926 19:37:12.362272 23503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:34227 every 8 connection(s)
I20260926 19:37:12.363646 23376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:37:12.364077 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23376
Sep 26 19:37:12 dist-test-slave-phbh krb5kdc[22472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:12.381875 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.365474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.6.215.254:42151)
0926 19:37:12.365693 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.365699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.366056 (+ 357us) server_negotiation.cc:407] Connection header received
0926 19:37:12.366273 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.366278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.366358 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.366468 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:12.368511 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.369674 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.370991 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.371642 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.371772 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.374800 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:12.374806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:12.374811 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:12.374851 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:12.378537 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.379529 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.379535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.379539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.379697 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.380081 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.380085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.380088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.380483 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:12.380694 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.381261 (+ 567us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.381487 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:37:12.387003 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.366364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:59493 (local address 127.6.215.254:42151)
0926 19:37:12.366902 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.366906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.367950 (+ 1044us) server_negotiation.cc:407] Connection header received
0926 19:37:12.370294 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.370299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.370378 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.370484 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:12.372367 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.373594 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.375287 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.375911 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.376037 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.380625 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:12.380636 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:12.380640 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:12.380691 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:12.383853 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.384721 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.384731 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.384734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.384891 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.385306 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.385312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.385315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.385564 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:12.385705 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.386395 (+ 690us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.386612 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":371,"threads_started":1}
I20260926 19:37:12.387974 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.367047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:59493)
0926 19:37:12.367812 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:12.367846 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:12.370052 (+ 2206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:12.370725 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:12.370733 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:12.371410 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:12.372172 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.372200 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.373735 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.373745 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.375149 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.375162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.376152 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.376159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.376466 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.377250 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:37:12.377288 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:12.380367 (+ 3079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:12.384004 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:12.384014 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:12.384037 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:12.384598 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:12.384996 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:12.385001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:12.385005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:12.385192 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:12.385697 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:12.385706 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:12.386050 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 19:37:12.386311 (+ 261us) client_negotiation.cc:240] Negotiation successful
0926 19:37:12.386862 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":326,"threads_started":1}
I20260926 19:37:12.389252 23504 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42151
I20260926 19:37:12.389555 23504 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:12.390265 23504 heartbeater.cc:511] Master 127.6.215.254:42151 requested a full tablet report, sending...
I20260926 19:37:12.391603 23055 ts_manager.cc:194] Registered new tserver with Master: 769c63f067aa4edc813dfa5a461a1235 (127.6.215.195:34227)
I20260926 19:37:12.392413 23055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:59493
I20260926 19:37:12.396696 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:37:12.403071 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.397572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.6.215.254:42151)
0926 19:37:12.397725 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.397729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.398237 (+ 508us) server_negotiation.cc:407] Connection header received
0926 19:37:12.398475 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.398482 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.398672 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.398812 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:37:12.399147 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.399866 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.401447 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.401849 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.401938 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.402359 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:37:12.402563 (+ 204us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 19:37:12.402881 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40674: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 19:37:12.403198 23507 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40674: token expired
W20260926 19:37:12.403679 22976 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.215.254:42151)
I20260926 19:37:12.416471 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.404125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.6.215.254:42151)
0926 19:37:12.404278 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.404284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.404361 (+ 77us) server_negotiation.cc:407] Connection header received
0926 19:37:12.404529 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.404534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.404598 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.404692 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:12.406167 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.406966 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.408248 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.408628 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.408786 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.411429 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:12.411434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:12.411437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:12.411467 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:12.413942 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.414576 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.414583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.414587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.414701 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.415063 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.415068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.415071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.415381 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:12.415648 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.415959 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.416165 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 19:37:12.419672 23055 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:37:12.426836 23055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:37:12.459118 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.438754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.6.215.193:38817)
0926 19:37:12.447525 (+ 8771us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.447539 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.447582 (+ 43us) server_negotiation.cc:407] Connection header received
0926 19:37:12.447710 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.447723 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.448064 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.448329 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:12.449611 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.452917 (+ 3306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.455486 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.457019 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.457199 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.457474 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.458676 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.458858 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8562,"thread_start_us":8396,"threads_started":1}
I20260926 19:37:12.459389 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.438533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:38817 (local address 127.0.0.1:53626)
0926 19:37:12.439468 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:12.439521 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:12.445756 (+ 6235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:12.448684 (+ 2928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:12.448697 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:12.448740 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:12.449324 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.449341 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.453081 (+ 3740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.453092 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.455313 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.455330 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.457273 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.457280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.457656 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.457784 (+ 128us) client_negotiation.cc:769] Sending connection context
0926 19:37:12.457943 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 19:37:12.458144 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":45,"thread_start_us":317,"threads_started":1}
I20260926 19:37:12.459662 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.439174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34227 (local address 127.0.0.1:60428)
0926 19:37:12.447136 (+ 7962us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:12.447162 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:12.447303 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:12.452230 (+ 4927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:12.452234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:12.452259 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:12.452648 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.452660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.454538 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.454542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.456731 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.456747 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.458823 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.458829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.458997 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.459030 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 19:37:12.459093 (+ 63us) client_negotiation.cc:240] Negotiation successful
0926 19:37:12.459227 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7785,"thread_start_us":7604,"threads_started":1}
I20260926 19:37:12.460997 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.439822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60428 (local address 127.6.215.195:34227)
0926 19:37:12.451666 (+ 11844us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.451674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.451707 (+ 33us) server_negotiation.cc:407] Connection header received
0926 19:37:12.451804 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.451812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.452042 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.452248 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:12.452882 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.454388 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.456920 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.458591 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.460128 (+ 1537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.460354 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.460422 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.460597 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11635,"thread_start_us":12676,"threads_started":1}
I20260926 19:37:12.461057 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.439636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:44077 (local address 127.0.0.1:33378)
0926 19:37:12.450946 (+ 11310us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:12.450964 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:12.451083 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:12.453702 (+ 2619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:12.453706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:12.453731 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:12.454367 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.454379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.456014 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.456022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.458005 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.458020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.460233 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.460239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.460426 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.460458 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 19:37:12.460544 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 19:37:12.460660 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11191,"mutex_wait_us":24,"thread_start_us":11056,"threads_started":1}
I20260926 19:37:12.461757 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.439917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.6.215.194:44077)
0926 19:37:12.453145 (+ 13228us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.453153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.453181 (+ 28us) server_negotiation.cc:407] Connection header received
0926 19:37:12.453271 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.453278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.453501 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.453684 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:12.454565 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.455886 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.458185 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.459706 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.459852 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.460091 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.461263 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.461447 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13044,"thread_start_us":12922,"threads_started":1}
I20260926 19:37:12.463583 23176 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7a16bce4ea4a789dda6137f91fddf1 (DEFAULT_TABLE table=test-table [id=4add950baebd47dea8c4a1b47cde9322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:12.464514 23439 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7a16bce4ea4a789dda6137f91fddf1 (DEFAULT_TABLE table=test-table [id=4add950baebd47dea8c4a1b47cde9322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:12.465202 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7a16bce4ea4a789dda6137f91fddf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:12.465668 23439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7a16bce4ea4a789dda6137f91fddf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:12.465442 23308 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7a16bce4ea4a789dda6137f91fddf1 (DEFAULT_TABLE table=test-table [id=4add950baebd47dea8c4a1b47cde9322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:12.466668 23308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7a16bce4ea4a789dda6137f91fddf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:12.470883 23241 heartbeater.cc:503] Master 127.6.215.254:42151 was elected leader, sending a full tablet report...
I20260926 19:37:12.477936 23520 tablet_bootstrap.cc:492] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Bootstrap starting.
I20260926 19:37:12.481407 23520 tablet_bootstrap.cc:654] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:12.482594 23522 tablet_bootstrap.cc:492] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Bootstrap starting.
I20260926 19:37:12.482743 23520 log.cc:824] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:12.486259 23522 tablet_bootstrap.cc:654] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:12.486375 23520 tablet_bootstrap.cc:492] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: No bootstrap required, opened a new log
I20260926 19:37:12.486624 23520 ts_tablet_manager.cc:1397] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 19:37:12.487500 23522 log.cc:824] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:12.490393 23522 tablet_bootstrap.cc:492] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: No bootstrap required, opened a new log
I20260926 19:37:12.490669 23522 ts_tablet_manager.cc:1397] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260926 19:37:12.492908 23521 tablet_bootstrap.cc:492] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Bootstrap starting.
I20260926 19:37:12.495725 23521 tablet_bootstrap.cc:654] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:12.496719 23521 log.cc:824] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:12.497087 23520 raft_consensus.cc:348] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.497634 23520 raft_consensus.cc:374] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:12.497740 23520 raft_consensus.cc:729] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 769c63f067aa4edc813dfa5a461a1235, State: Initialized, Role: FOLLOWER
I20260926 19:37:12.498430 23520 consensus_queue.cc:260] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.499574 23504 heartbeater.cc:503] Master 127.6.215.254:42151 was elected leader, sending a full tablet report...
I20260926 19:37:12.500022 23521 tablet_bootstrap.cc:492] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: No bootstrap required, opened a new log
I20260926 19:37:12.500207 23520 ts_tablet_manager.cc:1428] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260926 19:37:12.500308 23521 ts_tablet_manager.cc:1397] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260926 19:37:12.501128 23522 raft_consensus.cc:348] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.501634 23522 raft_consensus.cc:374] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:12.501781 23522 raft_consensus.cc:729] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e69e83e6ca46279a0cd10b2395390a, State: Initialized, Role: FOLLOWER
I20260926 19:37:12.502389 23522 consensus_queue.cc:260] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.503492 23373 heartbeater.cc:503] Master 127.6.215.254:42151 was elected leader, sending a full tablet report...
I20260926 19:37:12.505649 23522 ts_tablet_manager.cc:1428] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260926 19:37:12.509603 23521 raft_consensus.cc:348] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.509930 23521 raft_consensus.cc:374] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:12.510006 23521 raft_consensus.cc:729] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7071382e39f4bf1a300880c1e682f01, State: Initialized, Role: FOLLOWER
I20260926 19:37:12.510489 23521 consensus_queue.cc:260] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.512008 23521 ts_tablet_manager.cc:1428] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
W20260926 19:37:12.615427 23505 tablet.cc:2369] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:37:12.632186 23527 raft_consensus.cc:482] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:37:12.632525 23527 raft_consensus.cc:504] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.634168 23527 leader_election.cc:288] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817), 769c63f067aa4edc813dfa5a461a1235 (127.6.215.195:34227)
I20260926 19:37:12.640980 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.634531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:41461 (local address 127.6.215.193:38817)
0926 19:37:12.634802 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.634810 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.635634 (+ 824us) server_negotiation.cc:407] Connection header received
0926 19:37:12.635827 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.635831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.635886 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.635979 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:12.636581 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.637356 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.639146 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.639947 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.640158 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.640230 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.640585 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.640712 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260926 19:37:12.641007 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.634780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:43209 (local address 127.6.215.195:34227)
0926 19:37:12.635019 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.635026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.635541 (+ 515us) server_negotiation.cc:407] Connection header received
0926 19:37:12.635772 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.635776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.635836 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.635930 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:12.636482 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.637275 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.639133 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.639969 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.640131 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.640201 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.640647 (+ 446us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.640791 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:37:12.641079 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.634563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:38817 (local address 127.6.215.194:41461)
0926 19:37:12.635519 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:12.635536 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:12.635681 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:12.635991 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:12.635996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:12.636018 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:12.636434 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.636442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.637479 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.637490 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.638966 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.638978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.640167 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.640173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.640415 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.640447 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 19:37:12.640542 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 19:37:12.640658 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":1,"thread_start_us":630,"threads_started":1}
I20260926 19:37:12.641680 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.634518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34227 (local address 127.6.215.194:43209)
0926 19:37:12.635411 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:12.635431 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:12.635617 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:12.635946 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:12.635950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:12.635972 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:12.636338 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.636348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.637400 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.637406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.638990 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:12.639001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.640359 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.640364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:12.640466 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.640487 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 19:37:12.641240 (+ 753us) client_negotiation.cc:240] Negotiation successful
0926 19:37:12.641352 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":47,"thread_start_us":482,"threads_started":1}
I20260926 19:37:12.642199 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7a16bce4ea4a789dda6137f91fddf1" candidate_uuid: "88e69e83e6ca46279a0cd10b2395390a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7071382e39f4bf1a300880c1e682f01" is_pre_election: true
I20260926 19:37:12.642673 23196 raft_consensus.cc:2454] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88e69e83e6ca46279a0cd10b2395390a in term 0.
I20260926 19:37:12.642597 23459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7a16bce4ea4a789dda6137f91fddf1" candidate_uuid: "88e69e83e6ca46279a0cd10b2395390a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769c63f067aa4edc813dfa5a461a1235" is_pre_election: true
I20260926 19:37:12.643043 23459 raft_consensus.cc:2454] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88e69e83e6ca46279a0cd10b2395390a in term 0.
I20260926 19:37:12.643314 23262 leader_election.cc:302] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88e69e83e6ca46279a0cd10b2395390a, f7071382e39f4bf1a300880c1e682f01; no voters:
I20260926 19:37:12.643735 23527 raft_consensus.cc:2781] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260926 19:37:12.643781 23374 tablet.cc:2369] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:37:12.643908 23527 raft_consensus.cc:482] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:37:12.644038 23527 raft_consensus.cc:3037] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:12.646220 23527 raft_consensus.cc:504] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.646847 23527 leader_election.cc:288] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [CANDIDATE]: Term 1 election: Requested vote from peers f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817), 769c63f067aa4edc813dfa5a461a1235 (127.6.215.195:34227)
I20260926 19:37:12.647286 23196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7a16bce4ea4a789dda6137f91fddf1" candidate_uuid: "88e69e83e6ca46279a0cd10b2395390a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7071382e39f4bf1a300880c1e682f01"
I20260926 19:37:12.647279 23459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7a16bce4ea4a789dda6137f91fddf1" candidate_uuid: "88e69e83e6ca46279a0cd10b2395390a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "769c63f067aa4edc813dfa5a461a1235"
I20260926 19:37:12.647482 23459 raft_consensus.cc:3037] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:12.647482 23196 raft_consensus.cc:3037] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:12.650632 23459 raft_consensus.cc:2454] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88e69e83e6ca46279a0cd10b2395390a in term 1.
I20260926 19:37:12.650632 23196 raft_consensus.cc:2454] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88e69e83e6ca46279a0cd10b2395390a in term 1.
I20260926 19:37:12.651050 23263 leader_election.cc:302] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 769c63f067aa4edc813dfa5a461a1235, 88e69e83e6ca46279a0cd10b2395390a; no voters:
I20260926 19:37:12.651387 23527 raft_consensus.cc:2781] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:37:12.651804 23527 raft_consensus.cc:686] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 1 LEADER]: Becoming Leader. State: Replica: 88e69e83e6ca46279a0cd10b2395390a, State: Running, Role: LEADER
I20260926 19:37:12.652232 23527 consensus_queue.cc:237] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } }
I20260926 19:37:12.656733 23054 catalog_manager.cc:5495] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a reported cstate change: term changed from 0 to 1, leader changed from <none> to 88e69e83e6ca46279a0cd10b2395390a (127.6.215.194). New cstate: current_term: 1 leader_uuid: "88e69e83e6ca46279a0cd10b2395390a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88e69e83e6ca46279a0cd10b2395390a" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 } health_report { overall_health: UNKNOWN } } }
W20260926 19:37:12.684433 23242 tablet.cc:2369] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:37:12.698786 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.693512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.6.215.194:44077)
0926 19:37:12.693720 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.693726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.693744 (+ 18us) server_negotiation.cc:407] Connection header received
0926 19:37:12.693927 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.693930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.694013 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.694159 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:37:12.694523 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.695217 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.696586 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.697049 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.697191 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.697691 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:37:12.698172 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:37:12.698328 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.698430 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.698508 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:37:12.706796 23459 raft_consensus.cc:1261] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 1 FOLLOWER]: Refusing update from remote peer 88e69e83e6ca46279a0cd10b2395390a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:37:12.706825 23196 raft_consensus.cc:1261] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Refusing update from remote peer 88e69e83e6ca46279a0cd10b2395390a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:37:12.707686 23531 consensus_queue.cc:1035] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [LEADER]: Connected to new peer: Peer: permanent_uuid: "769c63f067aa4edc813dfa5a461a1235" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:37:12.707999 23527 consensus_queue.cc:1035] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7071382e39f4bf1a300880c1e682f01" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:37:12.725610 23536 mvcc.cc:205] Tried to move back new op lower bound from 7333689068356685824 to 7333689068147363840. Current Snapshot: MvccSnapshot[applied={T|T < 7333689068356685824}]
I20260926 19:37:12.740209 23054 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:37:12.740513 23054 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:37:12.743971 23054 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Sending DeleteTablet for 3 replicas of tablet 8d7a16bce4ea4a789dda6137f91fddf1
I20260926 19:37:12.745157 23308 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7a16bce4ea4a789dda6137f91fddf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:12 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:33378
I20260926 19:37:12.745289 23439 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7a16bce4ea4a789dda6137f91fddf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:12 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:60428
I20260926 19:37:12.745307 23176 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7a16bce4ea4a789dda6137f91fddf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:12 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:53626
I20260926 19:37:12.748075 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 23092
I20260926 19:37:12.755272 23546 tablet_replica.cc:330] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: stopping tablet replica
I20260926 19:37:12.756058 23546 raft_consensus.cc:2229] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:37:12.756713 23546 raft_consensus.cc:2258] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:12.758201 23545 tablet_replica.cc:330] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: stopping tablet replica
I20260926 19:37:12.758869 23544 tablet_replica.cc:330] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: stopping tablet replica
I20260926 19:37:12.759449 23545 raft_consensus.cc:2229] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:37:12.760031 23545 raft_consensus.cc:2258] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:12.760553 23544 raft_consensus.cc:2229] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:37:12.761196 23546 ts_tablet_manager.cc:1905] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:12.761896 23544 raft_consensus.cc:2258] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:12.762269 23545 ts_tablet_manager.cc:1905] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:12.769273 23545 ts_tablet_manager.cc:1918] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:12.769502 23545 log.cc:1196] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/8d7a16bce4ea4a789dda6137f91fddf1
I20260926 19:37:12.770066 23545 ts_tablet_manager.cc:1939] T 8d7a16bce4ea4a789dda6137f91fddf1 P 769c63f067aa4edc813dfa5a461a1235: Deleting consensus metadata
I20260926 19:37:12.771656 23544 ts_tablet_manager.cc:1905] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:12.772034 23041 catalog_manager.cc:4841] TS 769c63f067aa4edc813dfa5a461a1235 (127.6.215.195:34227): tablet 8d7a16bce4ea4a789dda6137f91fddf1 (table test-table [id=4add950baebd47dea8c4a1b47cde9322]) successfully deleted
I20260926 19:37:12.769390 23546 ts_tablet_manager.cc:1918] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:12.772758 23546 log.cc:1196] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/8d7a16bce4ea4a789dda6137f91fddf1
I20260926 19:37:12.773334 23546 ts_tablet_manager.cc:1939] T 8d7a16bce4ea4a789dda6137f91fddf1 P f7071382e39f4bf1a300880c1e682f01: Deleting consensus metadata
I20260926 19:37:12.774524 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.750200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.6.215.193:38817)
0926 19:37:12.750440 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.750448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.750471 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:37:12.750545 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.750553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.750664 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.750787 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:12.753569 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.755499 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.758663 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.759386 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.759520 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.763410 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:12.763424 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:12.763443 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:12.763493 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:12.767670 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.768671 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.768681 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.768685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.768899 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.769398 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.769404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.769407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.772222 (+ 2815us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:12.772539 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.773631 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.774098 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 19:37:12.775134 23040 catalog_manager.cc:4841] TS f7071382e39f4bf1a300880c1e682f01 (127.6.215.193:38817): tablet 8d7a16bce4ea4a789dda6137f91fddf1 (table test-table [id=4add950baebd47dea8c4a1b47cde9322]) successfully deleted
I20260926 19:37:12.776258 23236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:12.780378 23544 ts_tablet_manager.cc:1918] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:12.780725 23544 log.cc:1196] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/8d7a16bce4ea4a789dda6137f91fddf1
I20260926 19:37:12.781968 23544 ts_tablet_manager.cc:1939] T 8d7a16bce4ea4a789dda6137f91fddf1 P 88e69e83e6ca46279a0cd10b2395390a: Deleting consensus metadata
I20260926 19:37:12.784286 23040 catalog_manager.cc:4841] TS 88e69e83e6ca46279a0cd10b2395390a (127.6.215.194:44077): tablet 8d7a16bce4ea4a789dda6137f91fddf1 (table test-table [id=4add950baebd47dea8c4a1b47cde9322]) successfully deleted
I20260926 19:37:12.905964 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 23092
I20260926 19:37:12.923915 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 23245
I20260926 19:37:12.938823 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:12.924647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33388 (local address 127.6.215.194:44077)
0926 19:37:12.924878 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:12.924884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:12.924972 (+ 88us) server_negotiation.cc:407] Connection header received
0926 19:37:12.925185 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:12.925196 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:12.925257 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:12.925361 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:12.926666 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.927528 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.929329 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:12.929822 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:12.929983 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:12.932956 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:12.932968 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:12.932983 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:12.933028 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:12.935949 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.936649 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.936656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.936659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.936870 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:12.937254 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:12.937259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:12.937262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:12.937604 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:12.937952 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:12.938236 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 19:37:12.938444 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 19:37:12.939622 23368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:13.053013 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 23245
I20260926 19:37:13.071488 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 23376
I20260926 19:37:13.085175 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:13.072251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.6.215.195:34227)
0926 19:37:13.072439 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:13.072447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:13.072529 (+ 82us) server_negotiation.cc:407] Connection header received
0926 19:37:13.072755 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:13.072759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:13.072961 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:13.073068 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:13.074436 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:13.075160 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:13.076526 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:13.076968 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:13.077098 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:13.079882 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:13.079895 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:13.079910 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:13.079941 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:13.082655 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:13.083319 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:13.083326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:13.083328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:13.083462 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:13.083784 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:13.083787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:13.083789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:13.084091 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:13.084313 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:13.084564 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 19:37:13.084715 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 19:37:13.085816 23499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:13.173928 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 23376
I20260926 19:37:13.190582 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 23022
I20260926 19:37:13.191684 23085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:13.305871 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 23022
I20260926 19:37:33.327642 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42151
--webserver_interface=127.6.215.254
--webserver_port=44071
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:33.666961 23551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:33.667250 23551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:33.667339 23551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:33.674556 23551 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 19:37:33.674654 23551 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 19:37:33.674692 23551 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 19:37:33.674744 23551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:37:33.674798 23551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:33.674870 23551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:33.674921 23551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:37:33.674973 23551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:37:33.684800 23551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=44071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:33.686096 23551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:33.687362 23551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:33.699381 23556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:33.699514 23557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:33.699445 23559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:33.700333 23551 server_base.cc:1034] running on GCE node
I20260926 19:37:33.701184 23551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:33.702530 23551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:33.703738 23551 hybrid_clock.cc:648] HybridClock initialized: now 1790451453703661 us; error 53 us; skew 500 ppm
I20260926 19:37:33.704006 23551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:33 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451453, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:33.707142 23551 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:37:33.708608 23551 webserver.cc:460] Webserver started at http://127.6.215.254:44071/ using document root <none> and password file <none>
I20260926 19:37:33.709334 23551 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:33.709470 23551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:33.714715 23551 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260926 19:37:33.717737 23566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:33.719058 23551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:37:33.719250 23551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "2107f7ea91214f7698bf62e3c85274ff"
format_stamp: "Formatted at 2026-09-26 19:36:47 on dist-test-slave-phbh"
server_key: "7663e621ce5e2930b66a22ed4918d18e"
server_key_iv: "a54a72a46fe8be5db52918c4b5eb1528"
server_key_version: "encryptionkey@0"
I20260926 19:37:33.720206 23551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:33.734985 23551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:33.739315 23551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:33.739650 23551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:33.758265 23551 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42151
I20260926 19:37:33.758294 23617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42151 every 8 connection(s)
I20260926 19:37:33.759940 23551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:37:33.762023 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23551
I20260926 19:37:33.763033 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:38817
--local_ip_for_outbound_sockets=127.6.215.193
--tserver_master_addrs=127.6.215.254:42151
--webserver_port=38015
--webserver_interface=127.6.215.193
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:37:33.770298 23618 sys_catalog.cc:263] Verifying existing consensus state
I20260926 19:37:33.774537 23618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Bootstrap starting.
I20260926 19:37:33.794988 23618 log.cc:824] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:33.810973 23618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 19:37:33.811556 23618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Bootstrap complete.
I20260926 19:37:33.821161 23618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:33.822095 23618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2107f7ea91214f7698bf62e3c85274ff, State: Initialized, Role: FOLLOWER
I20260926 19:37:33.822762 23618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:33.822992 23618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:37:33.823114 23618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:37:33.823302 23618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 2 FOLLOWER]: Advancing to term 3
I20260926 19:37:33.827412 23618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:33.827927 23618 leader_election.cc:302] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2107f7ea91214f7698bf62e3c85274ff; no voters:
I20260926 19:37:33.828665 23618 leader_election.cc:288] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 19:37:33.828836 23622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 3 FOLLOWER]: Leader election won for term 3
I20260926 19:37:33.830021 23622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [term 3 LEADER]: Becoming Leader. State: Replica: 2107f7ea91214f7698bf62e3c85274ff, State: Running, Role: LEADER
I20260926 19:37:33.830646 23622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } }
I20260926 19:37:33.830871 23618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:37:33.833186 23624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2107f7ea91214f7698bf62e3c85274ff. Latest consensus state: current_term: 3 leader_uuid: "2107f7ea91214f7698bf62e3c85274ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } } }
I20260926 19:37:33.833921 23624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:33.833581 23623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2107f7ea91214f7698bf62e3c85274ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2107f7ea91214f7698bf62e3c85274ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42151 } } }
I20260926 19:37:33.834574 23623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:33.845834 23630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:37:33.854791 23630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:37:33.856328 23630 catalog_manager.cc:1259] Loaded cluster ID: bc8141f4e81b4fe9a756a120498284c9
I20260926 19:37:33.856488 23630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:37:33.860659 23630 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:37:33.862370 23630 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Loaded TSK: 0
I20260926 19:37:33.862622 23630 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2107f7ea91214f7698bf62e3c85274ff: Loaded TSK: 1
F20260926 19:37:33.863798 23630 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7feed3c099ad google::LogMessage::Fail() at ??:0
@ 0x7feed3c0c88a google::LogMessage::SendToLog() at ??:0
@ 0x7feed3c094de google::LogMessage::Flush() at ??:0
@ 0x7feed3c0aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7feed796d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7feed795c73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7feee96381c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7feee9637783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7feee96c32a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7feee96c30c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7feee970d5e9 std::function<>::operator()() at ??:0
@ 0x7feee963bcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7feee9639c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7feee96c1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7feee96c1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7feed6855345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7feed685891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7feed6858799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7feed6833eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7feed9b0d6db start_thread at ??:0
@ 0x7feed2aa361f clone at ??:0
W20260926 19:37:34.143750 23620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:34.144078 23620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:34.144246 23620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:34.151845 23620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:34.152009 23620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:34.152200 23620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:37:34.162226 23620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:38817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=38015
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:34.163614 23620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:34.165019 23620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:34.178599 23665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:34.178561 23668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:34.178647 23666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:34.182354 23620 server_base.cc:1034] running on GCE node
I20260926 19:37:34.183056 23620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:34.184209 23620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:34.185458 23620 hybrid_clock.cc:648] HybridClock initialized: now 1790451454185391 us; error 56 us; skew 500 ppm
I20260926 19:37:34.185830 23620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:34 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451454, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:34.189282 23620 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:37:34.190969 23620 webserver.cc:460] Webserver started at http://127.6.215.193:38015/ using document root <none> and password file <none>
I20260926 19:37:34.191700 23620 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:34.191854 23620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:34.197544 23620 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 19:37:34.200585 23675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:34.202160 23620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:37:34.202342 23620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "f7071382e39f4bf1a300880c1e682f01"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "0d6293577491651b569c627aa37122a0"
server_key_iv: "b782812b3500499ab94754aebb13e552"
server_key_version: "encryptionkey@0"
I20260926 19:37:34.203411 23620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:34.244987 23620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:34.249554 23620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:34.249843 23620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:34.250851 23620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:34.252205 23620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:34.252357 23620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:34.252483 23620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:34.252571 23620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:34.288450 23620 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:38817
I20260926 19:37:34.288543 23787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:38817 every 8 connection(s)
I20260926 19:37:34.290050 23620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:37:34.292234 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23620
I20260926 19:37:34.292934 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:44077
--local_ip_for_outbound_sockets=127.6.215.194
--tserver_master_addrs=127.6.215.254:42151
--webserver_port=34885
--webserver_interface=127.6.215.194
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:37:34.295756 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:34.293430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:42239)
0926 19:37:34.294323 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:34.294605 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":299,"threads_started":1}
W20260926 19:37:34.296543 23788 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:42151 (0 consecutive failures): Network error: Failed to ping master at 127.6.215.254:42151: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
I20260926 19:37:34.297523 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:34.297088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:59243)
0926 19:37:34.297277 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:34.297372 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 19:37:34.298516 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:34.298083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:36083)
0926 19:37:34.298258 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:34.298363 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 19:37:34.298780 23788 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 19:37:34.624234 23791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:34.624517 23791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:34.624670 23791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:34.631901 23791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:34.632001 23791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:34.632177 23791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:37:34.642004 23791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:44077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=34885
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:34.643261 23791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:34.644593 23791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:34.657658 23799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:34.657797 23796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:34.657532 23797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:34.658843 23791 server_base.cc:1034] running on GCE node
I20260926 19:37:34.659469 23791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:34.660420 23791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:34.661675 23791 hybrid_clock.cc:648] HybridClock initialized: now 1790451454661634 us; error 65 us; skew 500 ppm
I20260926 19:37:34.662000 23791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:34 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451454, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:34.665115 23791 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:37:34.666546 23791 webserver.cc:460] Webserver started at http://127.6.215.194:34885/ using document root <none> and password file <none>
I20260926 19:37:34.667217 23791 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:34.667367 23791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:34.672597 23791 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:37:34.675261 23806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:34.676566 23791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:37:34.676743 23791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "88e69e83e6ca46279a0cd10b2395390a"
format_stamp: "Formatted at 2026-09-26 19:36:48 on dist-test-slave-phbh"
server_key: "ff2399e8598e424126642adaf396e830"
server_key_iv: "7026d07a22866bc3063d3244e70c4372"
server_key_version: "encryptionkey@0"
I20260926 19:37:34.677784 23791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:34.719978 23791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:34.724324 23791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:34.724612 23791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:34.725656 23791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:34.726987 23791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:34.727108 23791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:34.727242 23791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:34.727329 23791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:34.761503 23791 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:44077
I20260926 19:37:34.761591 23918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:44077 every 8 connection(s)
I20260926 19:37:34.762928 23791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:37:34.767602 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:34.765827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:55039)
0926 19:37:34.766544 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:34.766770 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":261,"threads_started":1}
W20260926 19:37:34.768368 23919 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:42151 (0 consecutive failures): Network error: Failed to ping master at 127.6.215.254:42151: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
I20260926 19:37:34.769248 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:34.768939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:50161)
0926 19:37:34.769079 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:34.769147 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 19:37:34.769975 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23791
I20260926 19:37:34.770170 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:34.769822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:55943)
0926 19:37:34.769948 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:34.770018 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 19:37:34.770474 23919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 19:37:34.770710 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:34227
--local_ip_for_outbound_sockets=127.6.215.195
--tserver_master_addrs=127.6.215.254:42151
--webserver_port=36703
--webserver_interface=127.6.215.195
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:35.097118 23922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:35.097402 23922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:35.097502 23922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:35.104624 23922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:35.104733 23922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:35.104918 23922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:37:35.114674 23922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:34227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=36703
--tserver_master_addrs=127.6.215.254:42151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:35.115873 23922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:35.117108 23922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:35.130117 23927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:35.130097 23930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:35.130048 23928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:35.131150 23922 server_base.cc:1034] running on GCE node
I20260926 19:37:35.131703 23922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:35.132578 23922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:35.133769 23922 hybrid_clock.cc:648] HybridClock initialized: now 1790451455133712 us; error 47 us; skew 500 ppm
I20260926 19:37:35.134105 23922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:35 dist-test-slave-phbh krb5kdc[22472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451455, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:35.148303 23922 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:37:35.149907 23922 webserver.cc:460] Webserver started at http://127.6.215.195:36703/ using document root <none> and password file <none>
I20260926 19:37:35.150600 23922 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:35.150729 23922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:35.155990 23922 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260926 19:37:35.158700 23937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:35.159984 23922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:37:35.160151 23922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "769c63f067aa4edc813dfa5a461a1235"
format_stamp: "Formatted at 2026-09-26 19:36:49 on dist-test-slave-phbh"
server_key: "e95be1662df65f49e87eaf61d9b93e39"
server_key_iv: "9f421e53f765b414f30559526dc0a1ef"
server_key_version: "encryptionkey@0"
I20260926 19:37:35.161158 23922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:35.185441 23922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:35.190018 23922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:35.190306 23922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:35.191342 23922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:35.192880 23922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:35.193009 23922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:35.193141 23922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:35.193226 23922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:35.227876 23922 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:34227
I20260926 19:37:35.227970 24049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:34227 every 8 connection(s)
I20260926 19:37:35.229449 23922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:37:35.234428 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 23922
I20260926 19:37:35.234481 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:35.232351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:33277)
0926 19:37:35.233125 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:35.233375 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":269,"threads_started":1}
W20260926 19:37:35.235219 24050 heartbeater.cc:644] Failed to heartbeat to 127.6.215.254:42151 (0 consecutive failures): Network error: Failed to ping master at 127.6.215.254:42151: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
I20260926 19:37:35.236037 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:35.235703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:35363)
0926 19:37:35.235855 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:35.235933 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 19:37:35.237082 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:35.236729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:56919)
0926 19:37:35.236893 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:35.236975 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260926 19:37:35.237354 24050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 19:37:35.300124 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:35.299313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:46793)
0926 19:37:35.299779 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:35.299889 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":233,"threads_started":1}
I20260926 19:37:35.771873 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:35.771022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:48099)
0926 19:37:35.771521 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:35.771648 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":249,"threads_started":1}
I20260926 19:37:36.238739 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:36.237935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:55213)
0926 19:37:36.238415 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:36.238512 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":231,"threads_started":1}
I20260926 19:37:36.301877 24057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:36.301170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:34811)
0926 19:37:36.301635 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:36.301716 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":238,"threads_started":1}
I20260926 19:37:36.773586 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:36.772892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:49357)
0926 19:37:36.773343 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:36.773425 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":249,"threads_started":1}
I20260926 19:37:37.240275 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:37.239540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:46193)
0926 19:37:37.240004 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:37.240112 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":203,"threads_started":1}
I20260926 19:37:37.303454 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:37.302771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:53697)
0926 19:37:37.303193 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:37.303277 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":222,"threads_started":1}
I20260926 19:37:37.775130 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:37.774448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:36151)
0926 19:37:37.774860 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:37.774938 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":222,"threads_started":1}
I20260926 19:37:38.242209 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:38.241335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:40029)
0926 19:37:38.241885 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:38.242032 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":283,"threads_started":1}
I20260926 19:37:38.305229 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:38.304449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:41303)
0926 19:37:38.304966 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:38.305060 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":247,"threads_started":1}
I20260926 19:37:38.776844 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:38.775988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:45015)
0926 19:37:38.776518 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:38.776604 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":267,"threads_started":1}
I20260926 19:37:39.243855 24065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:39.243074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:56411)
0926 19:37:39.243591 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:39.243688 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":248,"threads_started":1}
I20260926 19:37:39.306903 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:39.306127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:52197)
0926 19:37:39.306654 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:39.306737 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":214,"threads_started":1}
I20260926 19:37:39.778530 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:39.777856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:34197)
0926 19:37:39.778276 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:39.778358 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":217,"threads_started":1}
I20260926 19:37:40.245586 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:40.244759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:34535)
0926 19:37:40.245332 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:40.245411 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":279,"threads_started":1}
I20260926 19:37:40.308650 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:40.307966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:38933)
0926 19:37:40.308398 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:40.308489 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":213,"threads_started":1}
I20260926 19:37:40.780102 24070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:40.779382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:54443)
0926 19:37:40.779815 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:40.779903 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":233,"threads_started":1}
I20260926 19:37:41.247241 24071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:41.246487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:55757)
0926 19:37:41.246982 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:41.247078 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":249,"threads_started":1}
I20260926 19:37:41.310418 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:41.309579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:40497)
0926 19:37:41.310110 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:41.310202 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":203,"threads_started":1}
I20260926 19:37:41.782169 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:41.781222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:34641)
0926 19:37:41.781829 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:41.781941 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":281,"threads_started":1}
I20260926 19:37:42.249315 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:42.248204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:59911)
0926 19:37:42.248816 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:42.248937 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":272,"threads_started":1}
I20260926 19:37:42.312107 24075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:42.311382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:51347)
0926 19:37:42.311824 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:42.311914 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":246,"threads_started":1}
I20260926 19:37:42.783747 24076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:42.783056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:46867)
0926 19:37:42.783490 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:42.783579 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":242,"threads_started":1}
I20260926 19:37:43.251080 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:43.250323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:45053)
0926 19:37:43.250806 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:43.250896 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":260,"threads_started":1}
I20260926 19:37:43.313880 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:43.313059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:39115)
0926 19:37:43.313614 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:43.313713 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":292,"threads_started":1}
I20260926 19:37:43.785723 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:43.784650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:37279)
0926 19:37:43.785299 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:43.785429 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":348,"threads_started":1}
I20260926 19:37:44.252861 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:44.252021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:37041)
0926 19:37:44.252515 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:44.252611 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":248,"threads_started":1}
I20260926 19:37:44.315486 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:44.314784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:47625)
0926 19:37:44.315243 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:44.315328 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":246,"threads_started":1}
I20260926 19:37:44.787326 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:44.786565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:55773)
0926 19:37:44.787076 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:44.787159 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":279,"threads_started":1}
I20260926 19:37:45.254492 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:45.253820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:58783)
0926 19:37:45.254252 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:45.254334 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":237,"threads_started":1}
I20260926 19:37:45.317255 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:45.316500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:53723)
0926 19:37:45.316991 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:45.317080 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":210,"threads_started":1}
I20260926 19:37:45.789032 24085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:45.788257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:56903)
0926 19:37:45.788711 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:45.788833 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":227,"threads_started":1}
I20260926 19:37:46.256122 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:46.255405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:56119)
0926 19:37:46.255815 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:46.255906 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":209,"threads_started":1}
I20260926 19:37:46.318998 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:46.318181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:46279)
0926 19:37:46.318675 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:46.318769 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":258,"threads_started":1}
I20260926 19:37:46.790813 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:46.790110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:59773)
0926 19:37:46.790572 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:46.790658 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":238,"threads_started":1}
I20260926 19:37:47.257908 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:47.257166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:52649)
0926 19:37:47.257643 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:47.257738 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":240,"threads_started":1}
I20260926 19:37:47.320655 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:47.319906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:47521)
0926 19:37:47.320352 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:47.320451 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":225,"threads_started":1}
I20260926 19:37:47.792536 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:47.791742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:53105)
0926 19:37:47.792244 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:47.792344 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":263,"threads_started":1}
I20260926 19:37:48.259645 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:48.258838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:47731)
0926 19:37:48.259350 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:48.259451 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":250,"threads_started":1}
I20260926 19:37:48.322320 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:48.321617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:40869)
0926 19:37:48.322060 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:48.322144 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":235,"threads_started":1}
I20260926 19:37:48.794457 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:48.793635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:60967)
0926 19:37:48.794161 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:48.794248 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":276,"threads_started":1}
I20260926 19:37:49.261370 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:49.260602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:58793)
0926 19:37:49.261102 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:49.261189 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":302,"threads_started":1}
I20260926 19:37:49.324101 24096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:49.323391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.193:57095)
0926 19:37:49.323800 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:49.323890 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":212,"threads_started":1}
I20260926 19:37:49.796298 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:49.795581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.194:59287)
0926 19:37:49.796004 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:49.796092 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":215,"threads_started":1}
I20260926 19:37:50.243759 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 23620
I20260926 19:37:50.261768 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:50.245011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46012 (local address 127.6.215.193:38817)
0926 19:37:50.245697 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:50.245735 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:50.245796 (+ 61us) server_negotiation.cc:407] Connection header received
0926 19:37:50.247587 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:50.247596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:50.248012 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:50.248301 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:50.249819 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:50.251005 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:50.252218 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:50.252960 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:50.253098 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:50.255741 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:50.255749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:50.255764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:50.255810 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:50.258749 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:50.259395 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:50.259402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:50.259408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:50.259575 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:50.259942 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:50.259945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:50.259947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:50.260437 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:50.260666 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:50.261084 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 19:37:50.261489 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":315,"threads_started":1}
I20260926 19:37:50.263324 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:50.262510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42151 (local address 127.6.215.195:40251)
0926 19:37:50.262941 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:50.263097 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.254:42151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":219,"threads_started":1}
I20260926 19:37:50.263535 23783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:50.356477 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 23620
I20260926 19:37:50.375552 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 23791
I20260926 19:37:50.394542 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:50.376626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.6.215.194:44077)
0926 19:37:50.377256 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:50.377271 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:50.377307 (+ 36us) server_negotiation.cc:407] Connection header received
0926 19:37:50.378880 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:50.378891 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:50.379321 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:50.379669 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:50.381269 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:50.382572 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:50.383798 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:50.384476 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:50.384671 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:50.387516 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:50.387534 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:50.387550 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:50.387637 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:50.391085 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:50.391781 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:50.391797 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:50.391804 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:50.391966 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:50.392371 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:50.392376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:50.392381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:50.392963 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:50.393272 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:50.393756 (+ 484us) server_negotiation.cc:299] Negotiation successful
0926 19:37:50.394263 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":330,"threads_started":1}
I20260926 19:37:50.395996 23914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:50.480417 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 23791
I20260926 19:37:50.499963 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 23922
I20260926 19:37:50.517865 24104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:50.501030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.6.215.195:34227)
0926 19:37:50.501577 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:50.501590 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:50.501630 (+ 40us) server_negotiation.cc:407] Connection header received
0926 19:37:50.503267 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:50.503276 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:50.503763 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:50.504121 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:50.505633 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:50.506814 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:50.508081 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:50.508731 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:50.508976 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:50.511634 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:50.511641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:50.511657 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:50.511693 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:50.514551 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:50.515256 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:50.515263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:50.515270 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:50.515428 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:50.515815 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:50.515821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:50.515838 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:50.516338 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:50.516642 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:50.517156 (+ 514us) server_negotiation.cc:299] Negotiation successful
0926 19:37:50.517574 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
I20260926 19:37:50.519281 24045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:50.603991 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 23922
2026-09-26T19:37:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64540 ms)
[----------] 30 tests from SecurityITest (294203 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:37:50 dist-test-slave-phbh krb5kdc[24108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:37:50 dist-test-slave-phbh krb5kdc[24108](info): set up 2 sockets
Sep 26 19:37:50 dist-test-slave-phbh krb5kdc[24108](info): commencing operation
krb5kdc: starting...
W20260926 19:37:51.713104 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:37:51 dist-test-slave-phbh krb5kdc[24108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:37:51.964463 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46149
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:46149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:52.292945 24121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:52.293223 24121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:52.293408 24121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:52.300956 24121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:37:52.301072 24121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:52.301170 24121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:52.301245 24121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:37:52.301311 24121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:37:52.312445 24121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:46149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:52.313845 24121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:52.315199 24121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:52.327999 24126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:52.328198 24121 server_base.cc:1034] running on GCE node
W20260926 19:37:52.328034 24127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:52.328431 24129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:52.329382 24121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:52.330819 24121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:52.332093 24121 hybrid_clock.cc:648] HybridClock initialized: now 1790451472332018 us; error 48 us; skew 500 ppm
I20260926 19:37:52.332456 24121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:52 dist-test-slave-phbh krb5kdc[24108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:52.335635 24121 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:37:52.337296 24121 webserver.cc:460] Webserver started at http://127.6.215.254:40359/ using document root <none> and password file <none>
I20260926 19:37:52.338099 24121 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:52.338270 24121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:52.338595 24121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:52.341830 24121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "fbc9c43795f649cbafac1a89c53ff4dd"
format_stamp: "Formatted at 2026-09-26 19:37:52 on dist-test-slave-phbh"
server_key: "8840278ce74012b0efcf9f2904cd4271"
server_key_iv: "21ff5830363b8e9998893f563a9ee4ea"
server_key_version: "encryptionkey@0"
I20260926 19:37:52.342620 24121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "fbc9c43795f649cbafac1a89c53ff4dd"
format_stamp: "Formatted at 2026-09-26 19:37:52 on dist-test-slave-phbh"
server_key: "8840278ce74012b0efcf9f2904cd4271"
server_key_iv: "21ff5830363b8e9998893f563a9ee4ea"
server_key_version: "encryptionkey@0"
I20260926 19:37:52.348145 24121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:37:52.351861 24136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:52.353428 24121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:37:52.353628 24121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "fbc9c43795f649cbafac1a89c53ff4dd"
format_stamp: "Formatted at 2026-09-26 19:37:52 on dist-test-slave-phbh"
server_key: "8840278ce74012b0efcf9f2904cd4271"
server_key_iv: "21ff5830363b8e9998893f563a9ee4ea"
server_key_version: "encryptionkey@0"
I20260926 19:37:52.353837 24121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:52.381567 24121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:52.385977 24121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:52.386284 24121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:52.404999 24121 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:46149
I20260926 19:37:52.405030 24187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:46149 every 8 connection(s)
I20260926 19:37:52.406350 24121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:37:52.409660 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 24121
I20260926 19:37:52.409873 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:37:52.410290 24188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:52.410588 7007 external_mini_cluster.cc:1428] Setting key a26a0da6cd6a389ac5e5b5032ee7685b
I20260926 19:37:52.422475 24188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd: Bootstrap starting.
Sep 26 19:37:52 dist-test-slave-phbh krb5kdc[24108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:52.425738 24188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:52.427217 24188 log.cc:824] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:52.430495 24188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd: No bootstrap required, opened a new log
I20260926 19:37:52.433346 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:52.412577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59410 (local address 127.6.215.254:46149)
0926 19:37:52.413451 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:52.413463 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:52.413505 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:37:52.415009 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:52.415028 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:52.415405 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:52.415786 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:52.417269 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:52.418518 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:52.419847 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:52.420558 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:52.420809 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:52.425051 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:52.425060 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:52.425081 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:52.425122 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:52.428834 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:52.429594 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:52.429602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:52.429609 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:52.429820 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:52.430207 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:52.430212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:52.430215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:52.430877 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:52.431234 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:52.431515 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 19:37:52.431941 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":456,"threads_started":1}
I20260926 19:37:52.439409 24188 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46149 } }
I20260926 19:37:52.439762 24188 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:52.439841 24188 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbc9c43795f649cbafac1a89c53ff4dd, State: Initialized, Role: FOLLOWER
I20260926 19:37:52.440378 24188 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46149 } }
I20260926 19:37:52.440582 24188 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:37:52.440727 24188 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:37:52.440979 24188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:52.443332 24188 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46149 } }
I20260926 19:37:52.443771 24188 leader_election.cc:302] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbc9c43795f649cbafac1a89c53ff4dd; no voters:
I20260926 19:37:52.444363 24188 leader_election.cc:288] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:37:52.444507 24193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:37:52.445364 24193 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [term 1 LEADER]: Becoming Leader. State: Replica: fbc9c43795f649cbafac1a89c53ff4dd, State: Running, Role: LEADER
I20260926 19:37:52.445906 24193 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46149 } }
I20260926 19:37:52.446470 24188 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:37:52.448956 24195 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbc9c43795f649cbafac1a89c53ff4dd. Latest consensus state: current_term: 1 leader_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46149 } } }
I20260926 19:37:52.449769 24195 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:52.449203 24194 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbc9c43795f649cbafac1a89c53ff4dd" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46149 } } }
I20260926 19:37:52.450412 24194 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:52.454769 24201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:37:52.460322 24201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:37:52.470149 24201 catalog_manager.cc:1347] Generated new cluster ID: d60f23dbf9e14f4da486acfae79dfdde
I20260926 19:37:52.470283 24201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:37:52.494627 24201 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:37:52.495975 24201 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:37:52.503242 24201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd: Generated new TSK 0
I20260926 19:37:52.504032 24201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:37:52.580672 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46149
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:52.912091 24214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:52.912386 24214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:52.912479 24214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:52.919764 24214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:52.919886 24214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:52.920001 24214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:37:52.929982 24214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:52.931226 24214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:52.932694 24214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:52.946394 24219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:52.946273 24222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:52.946257 24220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:52.946764 24214 server_base.cc:1034] running on GCE node
I20260926 19:37:52.947528 24214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:52.948503 24214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:52.949738 24214 hybrid_clock.cc:648] HybridClock initialized: now 1790451472949671 us; error 81 us; skew 500 ppm
I20260926 19:37:52.950062 24214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:52 dist-test-slave-phbh krb5kdc[24108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:52.953246 24214 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:37:52.954625 24214 webserver.cc:460] Webserver started at http://127.6.215.193:39137/ using document root <none> and password file <none>
I20260926 19:37:52.955320 24214 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:52.955502 24214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:52.955855 24214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:52.958775 24214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "c85ea12caad440e8ae98dfc4bedb9cca"
format_stamp: "Formatted at 2026-09-26 19:37:52 on dist-test-slave-phbh"
server_key: "4c0f228c3347bc714b54d044d92a3516"
server_key_iv: "deceeead87feeff3e1074d81365c4ae7"
server_key_version: "encryptionkey@0"
I20260926 19:37:52.959519 24214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "c85ea12caad440e8ae98dfc4bedb9cca"
format_stamp: "Formatted at 2026-09-26 19:37:52 on dist-test-slave-phbh"
server_key: "4c0f228c3347bc714b54d044d92a3516"
server_key_iv: "deceeead87feeff3e1074d81365c4ae7"
server_key_version: "encryptionkey@0"
I20260926 19:37:52.965758 24214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 19:37:52.969748 24229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:52.971225 24214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:37:52.971366 24214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "c85ea12caad440e8ae98dfc4bedb9cca"
format_stamp: "Formatted at 2026-09-26 19:37:52 on dist-test-slave-phbh"
server_key: "4c0f228c3347bc714b54d044d92a3516"
server_key_iv: "deceeead87feeff3e1074d81365c4ae7"
server_key_version: "encryptionkey@0"
I20260926 19:37:52.971542 24214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:52.992133 24214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:52.996517 24214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:52.996762 24214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:52.997748 24214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:52.999318 24214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:52.999394 24214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:52.999485 24214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:52.999531 24214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:53.033313 24214 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:42531
I20260926 19:37:53.033355 24341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:42531 every 8 connection(s)
I20260926 19:37:53.034639 24214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:37:53.039501 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 24214
I20260926 19:37:53.039750 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:37:53.040586 7007 external_mini_cluster.cc:1428] Setting key 662508a6196d965b617efa6ef3001f3c
Sep 26 19:37:53 dist-test-slave-phbh krb5kdc[24108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:53.059424 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:53.037308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:46585 (local address 127.6.215.254:46149)
0926 19:37:53.037798 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:53.037803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:53.038665 (+ 862us) server_negotiation.cc:407] Connection header received
0926 19:37:53.041445 (+ 2780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:53.041450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:53.041554 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:53.041680 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:53.043736 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.044601 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.047177 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.047716 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.047824 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:53.052280 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:53.052288 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:53.052294 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:53.052343 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:53.056059 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:53.056901 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:53.056909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:53.056912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:53.057045 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:53.057517 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:53.057522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:53.057524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:53.057846 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:53.058007 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:53.058737 (+ 730us) server_negotiation.cc:299] Negotiation successful
0926 19:37:53.058969 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":287,"threads_started":1}
I20260926 19:37:53.060392 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:53.037800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46149 (local address 127.6.215.193:46585)
0926 19:37:53.038488 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:53.038532 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:53.041104 (+ 2572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:53.041975 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:53.041987 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:53.042752 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:53.043543 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:53.043571 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.044794 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.044803 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:53.046967 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:53.046985 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.047983 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.047992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:53.048331 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:53.049215 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:37:53.049256 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:53.051970 (+ 2714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:53.056212 (+ 4242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:53.056218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:53.056233 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:53.056715 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:53.057176 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:53.057180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:53.057184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:53.057383 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:53.057979 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:53.057987 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:53.058355 (+ 368us) client_negotiation.cc:769] Sending connection context
0926 19:37:53.058639 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 19:37:53.059227 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":260,"threads_started":1}
I20260926 19:37:53.062136 24342 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46149
I20260926 19:37:53.062497 24342 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:53.063407 24342 heartbeater.cc:511] Master 127.6.215.254:46149 requested a full tablet report, sending...
I20260926 19:37:53.065732 24153 ts_manager.cc:194] Registered new tserver with Master: c85ea12caad440e8ae98dfc4bedb9cca (127.6.215.193:42531)
I20260926 19:37:53.067330 24153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:46585
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:37:53.099576 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46149
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:53.431833 24348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:53.432120 24348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:53.432274 24348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:53.439599 24348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:53.439733 24348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:53.439921 24348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:37:53.449928 24348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:53.451297 24348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:53.452615 24348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:53.466480 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:53.466593 24353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:53.466595 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:53.467672 24348 server_base.cc:1034] running on GCE node
I20260926 19:37:53.468327 24348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:53.469300 24348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:53.470556 24348 hybrid_clock.cc:648] HybridClock initialized: now 1790451473470499 us; error 37 us; skew 500 ppm
I20260926 19:37:53.470938 24348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:53 dist-test-slave-phbh krb5kdc[24108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:53.475323 24348 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:37:53.478168 24348 webserver.cc:460] Webserver started at http://127.6.215.194:46875/ using document root <none> and password file <none>
I20260926 19:37:53.479188 24348 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:53.479339 24348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:53.479720 24348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:53.483680 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "b2868883c65c431781ecd9f0b991b571"
format_stamp: "Formatted at 2026-09-26 19:37:53 on dist-test-slave-phbh"
server_key: "4a6bae8d93888af096c440e42e10f98c"
server_key_iv: "2d6183bd780cac2852232c1ec38a9fab"
server_key_version: "encryptionkey@0"
I20260926 19:37:53.484431 24348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "b2868883c65c431781ecd9f0b991b571"
format_stamp: "Formatted at 2026-09-26 19:37:53 on dist-test-slave-phbh"
server_key: "4a6bae8d93888af096c440e42e10f98c"
server_key_iv: "2d6183bd780cac2852232c1ec38a9fab"
server_key_version: "encryptionkey@0"
I20260926 19:37:53.489718 24348 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260926 19:37:53.493317 24363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:53.494788 24348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:37:53.495067 24348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "b2868883c65c431781ecd9f0b991b571"
format_stamp: "Formatted at 2026-09-26 19:37:53 on dist-test-slave-phbh"
server_key: "4a6bae8d93888af096c440e42e10f98c"
server_key_iv: "2d6183bd780cac2852232c1ec38a9fab"
server_key_version: "encryptionkey@0"
I20260926 19:37:53.495308 24348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:53.523264 24348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:53.527962 24348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:53.528268 24348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:53.529335 24348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:53.531028 24348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:53.531100 24348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:53.531250 24348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:53.531299 24348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:53.567420 24348 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:46583
I20260926 19:37:53.567459 24475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:46583 every 8 connection(s)
I20260926 19:37:53.569123 24348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:37:53.573484 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 24348
I20260926 19:37:53.573678 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:37:53.574116 7007 external_mini_cluster.cc:1428] Setting key 604184a7b9a2a0dabcee6ace043ad3a6
Sep 26 19:37:53 dist-test-slave-phbh krb5kdc[24108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:53.590781 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:53.571770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:56225 (local address 127.6.215.254:46149)
0926 19:37:53.572286 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:53.572291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:53.573097 (+ 806us) server_negotiation.cc:407] Connection header received
0926 19:37:53.575381 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:53.575387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:53.575487 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:53.575613 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:53.577389 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.578190 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.579905 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.580403 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.580559 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:53.584405 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:53.584410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:53.584414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:53.584450 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:53.587339 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:53.588167 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:53.588172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:53.588174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:53.588308 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:53.588853 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:53.588857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:53.588859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:53.589162 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:53.589339 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:53.590106 (+ 767us) server_negotiation.cc:299] Negotiation successful
0926 19:37:53.590361 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":286,"threads_started":1}
I20260926 19:37:53.591885 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:53.572156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46149 (local address 127.6.215.194:56225)
0926 19:37:53.572922 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:53.572966 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:53.575095 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:53.575861 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:53.575870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:53.576534 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:53.577210 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:53.577228 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.578362 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.578373 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:53.579747 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:53.579760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:53.580660 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:53.580668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:53.581020 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:53.581772 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:37:53.581798 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:53.584147 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:53.587533 (+ 3386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:53.587542 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:53.587561 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:53.588018 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:53.588439 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:53.588445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:53.588449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:53.588670 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:53.589305 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:53.589314 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:53.589711 (+ 397us) client_negotiation.cc:769] Sending connection context
0926 19:37:53.589997 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 19:37:53.590596 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":267,"threads_started":1}
I20260926 19:37:53.593616 24476 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46149
I20260926 19:37:53.593997 24476 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:53.594937 24476 heartbeater.cc:511] Master 127.6.215.254:46149 requested a full tablet report, sending...
I20260926 19:37:53.596637 24153 ts_manager.cc:194] Registered new tserver with Master: b2868883c65c431781ecd9f0b991b571 (127.6.215.194:46583)
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:37:53.597667 24153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:56225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:37:53.629077 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46149
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:33439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:53.951718 24482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:53.952013 24482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:53.952183 24482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:53.959661 24482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:53.959769 24482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:53.959973 24482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:37:53.969717 24482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:33439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:53.970988 24482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:53.972287 24482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:53.986488 24487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:53.986584 24488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:53.987073 24490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:53.987263 24482 server_base.cc:1034] running on GCE node
I20260926 19:37:53.987881 24482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:53.988889 24482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:53.990110 24482 hybrid_clock.cc:648] HybridClock initialized: now 1790451473990035 us; error 71 us; skew 500 ppm
I20260926 19:37:53.990413 24482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:53 dist-test-slave-phbh krb5kdc[24108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:53.993405 24482 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:37:53.994925 24482 webserver.cc:460] Webserver started at http://127.6.215.195:42171/ using document root <none> and password file <none>
I20260926 19:37:53.995618 24482 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:53.995762 24482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:53.996068 24482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:53.998989 24482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "0bc268c85a064a89bc4a89944ccc4501"
format_stamp: "Formatted at 2026-09-26 19:37:53 on dist-test-slave-phbh"
server_key: "41ae9d7a14d11997df6c357d26aec817"
server_key_iv: "38279718cdf0b93dda2829bbdfbdbb4b"
server_key_version: "encryptionkey@0"
I20260926 19:37:53.999814 24482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "0bc268c85a064a89bc4a89944ccc4501"
format_stamp: "Formatted at 2026-09-26 19:37:53 on dist-test-slave-phbh"
server_key: "41ae9d7a14d11997df6c357d26aec817"
server_key_iv: "38279718cdf0b93dda2829bbdfbdbb4b"
server_key_version: "encryptionkey@0"
I20260926 19:37:54.004917 24482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 19:37:54.008294 24497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:54.009586 24482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:37:54.009754 24482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "0bc268c85a064a89bc4a89944ccc4501"
format_stamp: "Formatted at 2026-09-26 19:37:53 on dist-test-slave-phbh"
server_key: "41ae9d7a14d11997df6c357d26aec817"
server_key_iv: "38279718cdf0b93dda2829bbdfbdbb4b"
server_key_version: "encryptionkey@0"
I20260926 19:37:54.009950 24482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:54.028440 24482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:54.032859 24482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:54.033157 24482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:54.034160 24482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:54.035636 24482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:54.035738 24482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:54.035883 24482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:54.035952 24482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:54.070475 24342 heartbeater.cc:503] Master 127.6.215.254:46149 was elected leader, sending a full tablet report...
I20260926 19:37:54.071203 24482 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:34089
I20260926 19:37:54.071224 24609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:34089 every 8 connection(s)
I20260926 19:37:54.072671 24482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:37:54.082448 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 24482
I20260926 19:37:54.082643 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:37:54.083127 7007 external_mini_cluster.cc:1428] Setting key 6b84b7503efb33bdf5461f570c84e23d
Sep 26 19:37:54 dist-test-slave-phbh krb5kdc[24108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:54.092315 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.075421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:49913 (local address 127.6.215.254:46149)
0926 19:37:54.075655 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.075659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.076810 (+ 1151us) server_negotiation.cc:407] Connection header received
0926 19:37:54.078629 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.078633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.078714 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.078816 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:54.080225 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.081061 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.082679 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.083121 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.083275 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.087238 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:54.087243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:54.087246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:54.087279 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:54.089771 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.090458 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.090462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.090464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.090589 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.090943 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.090946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.090948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.091205 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:54.091320 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.091900 (+ 580us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.092075 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 19:37:54.093494 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.075920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46149 (local address 127.6.215.195:49913)
0926 19:37:54.076630 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:54.076673 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:54.078365 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:54.078990 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:54.078997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:54.079512 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:54.080056 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.080073 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.081231 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.081235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.082484 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.082496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.083399 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.083406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.083710 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.084425 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:37:54.084467 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:54.087040 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:54.089896 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:54.089902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:54.089915 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:54.090358 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:54.090680 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:37:54.090683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:37:54.090685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:37:54.090853 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:37:54.091308 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:54.091313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:54.091609 (+ 296us) client_negotiation.cc:769] Sending connection context
0926 19:37:54.091828 (+ 219us) client_negotiation.cc:240] Negotiation successful
0926 19:37:54.092205 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":298,"threads_started":1}
I20260926 19:37:54.094903 24610 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46149
I20260926 19:37:54.095218 24610 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:54.095927 24610 heartbeater.cc:511] Master 127.6.215.254:46149 requested a full tablet report, sending...
I20260926 19:37:54.097608 24153 ts_manager.cc:194] Registered new tserver with Master: 0bc268c85a064a89bc4a89944ccc4501 (127.6.215.195:34089)
I20260926 19:37:54.098845 24153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:49913
I20260926 19:37:54.098901 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:37:54.116583 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.104275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.6.215.254:46149)
0926 19:37:54.104436 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.104440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.104888 (+ 448us) server_negotiation.cc:407] Connection header received
0926 19:37:54.105038 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.105041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.105129 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.105214 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:54.106743 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.107445 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.108825 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.109273 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.109370 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.111739 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:54.111744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:54.111747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:54.111775 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:54.114308 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.114967 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.114971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.114973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.115101 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.115431 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.115434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.115436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.115724 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:54.115913 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.116186 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.116344 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 19:37:54.120347 24153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:37:54.127315 24153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:37:54.166013 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.139491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.6.215.193:42531)
0926 19:37:54.158101 (+ 18610us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.158110 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.158133 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:37:54.158207 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.158213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.158444 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.158697 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:54.159721 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.161387 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.163768 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.164695 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.164903 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.165165 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.165486 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.165677 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18426,"thread_start_us":18270,"threads_started":1}
I20260926 19:37:54.166026 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.139176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:42531 (local address 127.0.0.1:34292)
0926 19:37:54.155312 (+ 16136us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:54.155379 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:54.155598 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:54.159009 (+ 3411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:54.159016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:54.159047 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:54.159498 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.159512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.161558 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.161562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.163611 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.163626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.164960 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.164967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.165203 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.165270 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 19:37:54.165421 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 19:37:54.165619 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15788,"mutex_wait_us":1,"thread_start_us":15608,"threads_started":1}
I20260926 19:37:54.167888 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.139340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34089 (local address 127.0.0.1:32828)
0926 19:37:54.157691 (+ 18351us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:54.157784 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:54.157926 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:54.160105 (+ 2179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:54.160112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:54.160142 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:54.160518 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.160529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.162579 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.162584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.165143 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.165158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.167007 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.167014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.167198 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.167229 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:37:54.167312 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 19:37:54.167459 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18200,"mutex_wait_us":124,"thread_start_us":18061,"threads_started":1}
I20260926 19:37:54.168284 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.139774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:46583 (local address 127.0.0.1:56904)
0926 19:37:54.160063 (+ 20289us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:54.160086 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:54.160211 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:54.161032 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:54.161037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:54.161065 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:54.161377 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.161386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.163234 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.163244 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.165455 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.165467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.167467 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.167472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.167657 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.167689 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 19:37:54.167778 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 19:37:54.167933 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20169,"thread_start_us":19995,"threads_started":1}
I20260926 19:37:54.168633 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.140153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.6.215.195:34089)
0926 19:37:54.157207 (+ 17054us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.157221 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.158063 (+ 842us) server_negotiation.cc:407] Connection header received
0926 19:37:54.158177 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.158185 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.158444 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.159823 (+ 1379us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:54.160720 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.162403 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.165305 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.166791 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.166919 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.167157 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.167690 (+ 533us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.167869 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16849,"thread_start_us":16686,"threads_started":1}
I20260926 19:37:54.169620 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.140032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.6.215.194:46583)
0926 19:37:54.160391 (+ 20359us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.160399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.160425 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:37:54.160515 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.160528 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.160777 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.161108 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:54.161528 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.163048 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.165623 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.167020 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.167157 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.167333 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.168377 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.168543 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20194,"thread_start_us":20259,"threads_started":1}
I20260926 19:37:54.171335 24277 tablet_service.cc:1467] Processing CreateTablet for tablet e2c7ff2160a4407e8506fba00643d244 (DEFAULT_TABLE table=test-table [id=b70ea7bdcaad487baebada62d295ea05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:54.172009 24545 tablet_service.cc:1467] Processing CreateTablet for tablet e2c7ff2160a4407e8506fba00643d244 (DEFAULT_TABLE table=test-table [id=b70ea7bdcaad487baebada62d295ea05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:54.172927 24277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2c7ff2160a4407e8506fba00643d244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:54.172578 24411 tablet_service.cc:1467] Processing CreateTablet for tablet e2c7ff2160a4407e8506fba00643d244 (DEFAULT_TABLE table=test-table [id=b70ea7bdcaad487baebada62d295ea05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:54.173456 24545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2c7ff2160a4407e8506fba00643d244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:54.173833 24411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2c7ff2160a4407e8506fba00643d244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:54.186846 24628 tablet_bootstrap.cc:492] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Bootstrap starting.
I20260926 19:37:54.188939 24629 tablet_bootstrap.cc:492] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Bootstrap starting.
I20260926 19:37:54.190726 24628 tablet_bootstrap.cc:654] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:54.192018 24628 log.cc:824] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:54.192337 24629 tablet_bootstrap.cc:654] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:54.193558 24630 tablet_bootstrap.cc:492] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Bootstrap starting.
I20260926 19:37:54.193667 24629 log.cc:824] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:54.196346 24629 tablet_bootstrap.cc:492] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: No bootstrap required, opened a new log
I20260926 19:37:54.196552 24628 tablet_bootstrap.cc:492] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: No bootstrap required, opened a new log
I20260926 19:37:54.196647 24629 ts_tablet_manager.cc:1397] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 19:37:54.196838 24628 ts_tablet_manager.cc:1397] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 19:37:54.197678 24630 tablet_bootstrap.cc:654] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:54.198905 24630 log.cc:824] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:54.202124 24630 tablet_bootstrap.cc:492] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: No bootstrap required, opened a new log
I20260926 19:37:54.202382 24630 ts_tablet_manager.cc:1397] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 19:37:54.204762 24629 raft_consensus.cc:348] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.205248 24629 raft_consensus.cc:374] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:54.205344 24629 raft_consensus.cc:729] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85ea12caad440e8ae98dfc4bedb9cca, State: Initialized, Role: FOLLOWER
I20260926 19:37:54.206024 24629 consensus_queue.cc:260] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.207392 24628 raft_consensus.cc:348] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.207792 24629 ts_tablet_manager.cc:1428] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Time spent starting tablet: real 0.011s user 0.001s sys 0.011s
I20260926 19:37:54.207809 24628 raft_consensus.cc:374] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:54.207901 24628 raft_consensus.cc:729] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2868883c65c431781ecd9f0b991b571, State: Initialized, Role: FOLLOWER
I20260926 19:37:54.208561 24628 consensus_queue.cc:260] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.209642 24476 heartbeater.cc:503] Master 127.6.215.254:46149 was elected leader, sending a full tablet report...
I20260926 19:37:54.210263 24628 ts_tablet_manager.cc:1428] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260926 19:37:54.212927 24630 raft_consensus.cc:348] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.213404 24630 raft_consensus.cc:374] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:54.213514 24630 raft_consensus.cc:729] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc268c85a064a89bc4a89944ccc4501, State: Initialized, Role: FOLLOWER
I20260926 19:37:54.214130 24630 consensus_queue.cc:260] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.215117 24610 heartbeater.cc:503] Master 127.6.215.254:46149 was elected leader, sending a full tablet report...
I20260926 19:37:54.215767 24630 ts_tablet_manager.cc:1428] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260926 19:37:54.287106 24343 tablet.cc:2369] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:37:54.321297 24477 tablet.cc:2369] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:37:54.324524 24611 tablet.cc:2369] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:37:54.331528 24634 raft_consensus.cc:482] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:37:54.331837 24634 raft_consensus.cc:504] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.333665 24634 leader_election.cc:288] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bc268c85a064a89bc4a89944ccc4501 (127.6.215.195:34089), b2868883c65c431781ecd9f0b991b571 (127.6.215.194:46583)
I20260926 19:37:54.341289 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.334225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:37443 (local address 127.6.215.195:34089)
0926 19:37:54.334471 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.334478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.335328 (+ 850us) server_negotiation.cc:407] Connection header received
0926 19:37:54.335511 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.335518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.335593 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.335667 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:54.336305 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.337259 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.339268 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.340242 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.340403 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.340494 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.340821 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.341030 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 19:37:54.341290 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.334340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:38395 (local address 127.6.215.194:46583)
0926 19:37:54.334562 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.334570 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.335251 (+ 681us) server_negotiation.cc:407] Connection header received
0926 19:37:54.335552 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.335556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.335614 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.335718 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:54.336458 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.337239 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.339479 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.340256 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.340436 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.340507 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.340874 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.341016 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 19:37:54.341614 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.334130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:34089 (local address 127.6.215.193:37443)
0926 19:37:54.335242 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:54.335256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:54.335365 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:54.335708 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:54.335713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:54.335736 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:54.336129 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.336138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.337415 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.337418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.339083 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.339096 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.340462 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.340468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.340641 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.340665 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 19:37:54.340739 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 19:37:54.340914 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"mutex_wait_us":61,"thread_start_us":666,"threads_started":1}
I20260926 19:37:54.341665 24637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.333946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:46583 (local address 127.6.215.193:38395)
0926 19:37:54.335132 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:54.335149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:54.335344 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:54.335768 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:54.335773 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:54.335804 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:54.336305 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.336319 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.337389 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.337395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.339113 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:54.339127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.340521 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.340528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:54.340661 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.340683 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 19:37:54.340845 (+ 162us) client_negotiation.cc:240] Negotiation successful
0926 19:37:54.340980 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":24,"thread_start_us":803,"threads_started":1}
I20260926 19:37:54.342816 24431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c7ff2160a4407e8506fba00643d244" candidate_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2868883c65c431781ecd9f0b991b571" is_pre_election: true
I20260926 19:37:54.343258 24431 raft_consensus.cc:2454] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c85ea12caad440e8ae98dfc4bedb9cca in term 0.
I20260926 19:37:54.343948 24232 leader_election.cc:302] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2868883c65c431781ecd9f0b991b571, c85ea12caad440e8ae98dfc4bedb9cca; no voters:
I20260926 19:37:54.343976 24565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c7ff2160a4407e8506fba00643d244" candidate_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc268c85a064a89bc4a89944ccc4501" is_pre_election: true
I20260926 19:37:54.344365 24634 raft_consensus.cc:2781] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:37:54.344480 24634 raft_consensus.cc:482] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:37:54.344551 24634 raft_consensus.cc:3037] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:54.344472 24565 raft_consensus.cc:2454] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c85ea12caad440e8ae98dfc4bedb9cca in term 0.
I20260926 19:37:54.347210 24634 raft_consensus.cc:504] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.347884 24634 leader_election.cc:288] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [CANDIDATE]: Term 1 election: Requested vote from peers 0bc268c85a064a89bc4a89944ccc4501 (127.6.215.195:34089), b2868883c65c431781ecd9f0b991b571 (127.6.215.194:46583)
I20260926 19:37:54.348400 24565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c7ff2160a4407e8506fba00643d244" candidate_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc268c85a064a89bc4a89944ccc4501"
I20260926 19:37:54.348423 24431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2c7ff2160a4407e8506fba00643d244" candidate_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2868883c65c431781ecd9f0b991b571"
I20260926 19:37:54.348598 24565 raft_consensus.cc:3037] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:54.348678 24431 raft_consensus.cc:3037] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:54.350821 24565 raft_consensus.cc:2454] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c85ea12caad440e8ae98dfc4bedb9cca in term 1.
I20260926 19:37:54.350870 24431 raft_consensus.cc:2454] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c85ea12caad440e8ae98dfc4bedb9cca in term 1.
I20260926 19:37:54.351245 24231 leader_election.cc:302] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc268c85a064a89bc4a89944ccc4501, c85ea12caad440e8ae98dfc4bedb9cca; no voters:
I20260926 19:37:54.351588 24634 raft_consensus.cc:2781] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:37:54.352114 24634 raft_consensus.cc:686] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 1 LEADER]: Becoming Leader. State: Replica: c85ea12caad440e8ae98dfc4bedb9cca, State: Running, Role: LEADER
I20260926 19:37:54.352699 24634 consensus_queue.cc:237] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } }
I20260926 19:37:54.357415 24153 catalog_manager.cc:5495] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca reported cstate change: term changed from 0 to 1, leader changed from <none> to c85ea12caad440e8ae98dfc4bedb9cca (127.6.215.193). New cstate: current_term: 1 leader_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c85ea12caad440e8ae98dfc4bedb9cca" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 42531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 } health_report { overall_health: UNKNOWN } } }
I20260926 19:37:54.405515 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.399307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.6.215.193:42531)
0926 19:37:54.399578 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.399582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.399603 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:37:54.399804 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.399808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.399906 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.400030 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:37:54.400393 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.401234 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.403105 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.403500 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.403613 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.404163 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:37:54.404866 (+ 703us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:37:54.404994 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.405153 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.405224 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:37:54.416496 24431 raft_consensus.cc:1261] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 1 FOLLOWER]: Refusing update from remote peer c85ea12caad440e8ae98dfc4bedb9cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:37:54.417124 24565 raft_consensus.cc:1261] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 1 FOLLOWER]: Refusing update from remote peer c85ea12caad440e8ae98dfc4bedb9cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:37:54.418392 24634 consensus_queue.cc:1035] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc268c85a064a89bc4a89944ccc4501" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:37:54.419157 24639 consensus_queue.cc:1035] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2868883c65c431781ecd9f0b991b571" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 46583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:37:54.455824 24153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:37:54.456046 24153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:37:54.459967 24153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbc9c43795f649cbafac1a89c53ff4dd: Sending DeleteTablet for 3 replicas of tablet e2c7ff2160a4407e8506fba00643d244
I20260926 19:37:54.461297 24545 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c7ff2160a4407e8506fba00643d244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:54 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:32828
I20260926 19:37:54.461306 24277 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c7ff2160a4407e8506fba00643d244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:54 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:34292
I20260926 19:37:54.462072 24411 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c7ff2160a4407e8506fba00643d244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:54 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:56904
I20260926 19:37:54.462782 24652 tablet_replica.cc:330] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: stopping tablet replica
I20260926 19:37:54.463095 24651 tablet_replica.cc:330] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: stopping tablet replica
I20260926 19:37:54.464076 24653 tablet_replica.cc:330] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: stopping tablet replica
I20260926 19:37:54.464484 24653 raft_consensus.cc:2229] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:37:54.464654 24652 raft_consensus.cc:2229] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:37:54.465441 24652 raft_consensus.cc:2258] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:54.465602 24653 raft_consensus.cc:2258] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:54.465804 24651 raft_consensus.cc:2229] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:37:54.466305 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 24214
I20260926 19:37:54.466302 24651 raft_consensus.cc:2258] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:54.468119 24653 ts_tablet_manager.cc:1905] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:54.468667 24652 ts_tablet_manager.cc:1905] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:54.468968 24651 ts_tablet_manager.cc:1905] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:54.475317 24653 ts_tablet_manager.cc:1918] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:54.475481 24653 log.cc:1196] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/e2c7ff2160a4407e8506fba00643d244
I20260926 19:37:54.476034 24653 ts_tablet_manager.cc:1939] T e2c7ff2160a4407e8506fba00643d244 P b2868883c65c431781ecd9f0b991b571: Deleting consensus metadata
I20260926 19:37:54.476090 24652 ts_tablet_manager.cc:1918] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:54.476423 24652 log.cc:1196] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/e2c7ff2160a4407e8506fba00643d244
I20260926 19:37:54.477037 24652 ts_tablet_manager.cc:1939] T e2c7ff2160a4407e8506fba00643d244 P c85ea12caad440e8ae98dfc4bedb9cca: Deleting consensus metadata
I20260926 19:37:54.477104 24651 ts_tablet_manager.cc:1918] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:54.477303 24651 log.cc:1196] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/e2c7ff2160a4407e8506fba00643d244
I20260926 19:37:54.477916 24651 ts_tablet_manager.cc:1939] T e2c7ff2160a4407e8506fba00643d244 P 0bc268c85a064a89bc4a89944ccc4501: Deleting consensus metadata
I20260926 19:37:54.478039 24139 catalog_manager.cc:4841] TS b2868883c65c431781ecd9f0b991b571 (127.6.215.194:46583): tablet e2c7ff2160a4407e8506fba00643d244 (table test-table [id=b70ea7bdcaad487baebada62d295ea05]) successfully deleted
I20260926 19:37:54.478864 24137 catalog_manager.cc:4841] TS c85ea12caad440e8ae98dfc4bedb9cca (127.6.215.193:42531): tablet e2c7ff2160a4407e8506fba00643d244 (table test-table [id=b70ea7bdcaad487baebada62d295ea05]) successfully deleted
I20260926 19:37:54.479817 24138 catalog_manager.cc:4841] TS 0bc268c85a064a89bc4a89944ccc4501 (127.6.215.195:34089): tablet e2c7ff2160a4407e8506fba00643d244 (table test-table [id=b70ea7bdcaad487baebada62d295ea05]) successfully deleted
Sep 26 19:37:54 dist-test-slave-phbh krb5kdc[24108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:37:54.498500 24624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.467920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.6.215.193:42531)
0926 19:37:54.468157 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.468167 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.481297 (+ 13130us) server_negotiation.cc:407] Connection header received
0926 19:37:54.481450 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.481455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.481547 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.481677 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:54.482887 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.484124 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.485597 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.486305 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.486442 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.490900 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:54.490920 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:54.490945 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:54.490995 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:54.494976 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.495820 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.495830 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.495834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.496114 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.496550 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.496556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.496560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.497101 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:54.497638 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.497848 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.498093 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 19:37:54.499359 24337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:54.597270 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 24214
I20260926 19:37:54.612324 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 24348
Sep 26 19:37:54 dist-test-slave-phbh krb5kdc[24108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:37:54.627461 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.613166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.6.215.194:46583)
0926 19:37:54.613363 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.613369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.613446 (+ 77us) server_negotiation.cc:407] Connection header received
0926 19:37:54.613588 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.613592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.613673 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.613757 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:54.615082 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.615813 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.617300 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.617679 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.617779 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.621967 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:54.622020 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:54.622038 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:54.622080 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:54.624962 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.625588 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.625595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.625597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.625707 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.626145 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.626149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.626151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.626532 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:54.626764 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.627019 (+ 255us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.627200 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:37:54.628125 24471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:54.718153 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 24348
I20260926 19:37:54.734530 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 24482
Sep 26 19:37:54 dist-test-slave-phbh krb5kdc[24108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:37:54.750136 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:54.735322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.6.215.195:34089)
0926 19:37:54.735508 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:54.735515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:54.735537 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:37:54.735773 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:54.735777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:54.735843 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:54.735951 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:54.737353 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.738067 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.739390 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:54.739775 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:54.739910 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:54.744399 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:54.744414 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:54.744430 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:54.744467 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:54.747100 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.747838 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.747846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.747848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.747990 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:54.748377 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:54.748382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:54.748386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:54.748728 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:54.749277 (+ 549us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:54.749568 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 19:37:54.749784 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 19:37:54.750813 24605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:54.839268 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 24482
I20260926 19:37:54.855899 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 24121
I20260926 19:37:54.857179 24183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:54.967211 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 24121
2026-09-26T19:37:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4372 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T19:37:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:37:55Z Disabled control of system clock
I20260926 19:37:55.049854 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42645
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:37849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 19:37:55.378924 24662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:55.379221 24662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:55.379371 24662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:55.386775 24662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:37:55.386889 24662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:55.386967 24662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:55.387059 24662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:37:55.387121 24662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:37:55.396519 24662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:55.398020 24662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:55.399822 24662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:55.413818 24668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:55.414335 24670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:55.413945 24667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:55.415540 24662 server_base.cc:1034] running on GCE node
I20260926 19:37:55.416706 24662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:55.418295 24662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:55.419552 24662 hybrid_clock.cc:648] HybridClock initialized: now 1790451475419481 us; error 58 us; skew 500 ppm
I20260926 19:37:55.419931 24662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:37:55.429275 24662 webserver.cc:460] Webserver started at http://127.6.215.254:42913/ using document root <none> and password file <none>
I20260926 19:37:55.430102 24662 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:55.430255 24662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:55.430603 24662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:55.433774 24662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "76b1425c5f334e2eb965429014f66039"
format_stamp: "Formatted at 2026-09-26 19:37:55 on dist-test-slave-phbh"
server_key: "de85cb1a47a1d30736f85bc719d1ae03"
server_key_iv: "1c3393e417da5811917c39649c3192c8"
server_key_version: "encryptionkey@0"
I20260926 19:37:55.434741 24662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "76b1425c5f334e2eb965429014f66039"
format_stamp: "Formatted at 2026-09-26 19:37:55 on dist-test-slave-phbh"
server_key: "de85cb1a47a1d30736f85bc719d1ae03"
server_key_iv: "1c3393e417da5811917c39649c3192c8"
server_key_version: "encryptionkey@0"
I20260926 19:37:55.439941 24662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 19:37:55.443588 24676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:55.445096 24662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:37:55.445295 24662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "76b1425c5f334e2eb965429014f66039"
format_stamp: "Formatted at 2026-09-26 19:37:55 on dist-test-slave-phbh"
server_key: "de85cb1a47a1d30736f85bc719d1ae03"
server_key_iv: "1c3393e417da5811917c39649c3192c8"
server_key_version: "encryptionkey@0"
I20260926 19:37:55.445523 24662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:55.465926 24662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:55.466848 24662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:55.467163 24662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:55.486492 24662 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42645
I20260926 19:37:55.486515 24727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42645 every 8 connection(s)
I20260926 19:37:55.487929 24662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:37:55.491678 24728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:55.494136 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 24662
I20260926 19:37:55.494308 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:37:55.494864 7007 external_mini_cluster.cc:1428] Setting key f4afe1306d8bf92d1cd271ed33fb8429
I20260926 19:37:55.503780 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039: Bootstrap starting.
I20260926 19:37:55.506995 24728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:55.508186 24728 log.cc:824] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:55.508576 24731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:55.496908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.6.215.254:42645)
0926 19:37:55.497969 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:55.497980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:55.498026 (+ 46us) server_negotiation.cc:407] Connection header received
0926 19:37:55.499752 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:55.499774 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:55.500226 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:55.500626 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:55.501263 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:55.502562 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:55.504078 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:55.504763 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:55.505025 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:55.506232 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:55.506243 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:55.506258 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:55.506793 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:55.506882 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:55.506907 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:55.507065 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:55.507242 (+ 177us) server_negotiation.cc:299] Negotiation successful
0926 19:37:55.507573 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":348,"threads_started":1}
I20260926 19:37:55.511155 24728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039: No bootstrap required, opened a new log
I20260926 19:37:55.518827 24728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1425c5f334e2eb965429014f66039" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:37:55.519222 24728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:55.519344 24728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b1425c5f334e2eb965429014f66039, State: Initialized, Role: FOLLOWER
I20260926 19:37:55.519975 24728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1425c5f334e2eb965429014f66039" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:37:55.520431 24728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:37:55.520656 24728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:37:55.520901 24728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:55.523272 24728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1425c5f334e2eb965429014f66039" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:37:55.523726 24728 leader_election.cc:302] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76b1425c5f334e2eb965429014f66039; no voters:
I20260926 19:37:55.524289 24728 leader_election.cc:288] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:37:55.524462 24733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:37:55.525243 24733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [term 1 LEADER]: Becoming Leader. State: Replica: 76b1425c5f334e2eb965429014f66039, State: Running, Role: LEADER
I20260926 19:37:55.525769 24733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1425c5f334e2eb965429014f66039" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } }
I20260926 19:37:55.526451 24728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:37:55.528689 24734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76b1425c5f334e2eb965429014f66039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1425c5f334e2eb965429014f66039" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } } }
I20260926 19:37:55.528815 24735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76b1425c5f334e2eb965429014f66039. Latest consensus state: current_term: 1 leader_uuid: "76b1425c5f334e2eb965429014f66039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76b1425c5f334e2eb965429014f66039" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42645 } } }
I20260926 19:37:55.529162 24734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:55.529212 24735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039 [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:55.531510 24738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:37:55.537195 24738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:37:55.546797 24738 catalog_manager.cc:1347] Generated new cluster ID: e220c811a6164e8d90e5acadc2a0610c
I20260926 19:37:55.546940 24738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:37:55.561390 24738 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:37:55.563189 24738 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:37:55.573155 24738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039: Generated new TSK 0
I20260926 19:37:55.574218 24738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:37:55.583700 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42645
--builtin_ntp_servers=127.6.215.212:37849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 19:37:55.912880 24752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:55.913213 24752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:55.913364 24752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:55.920641 24752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:55.920778 24752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:55.920991 24752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:37:55.930408 24752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:55.931769 24752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:55.933704 24752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:55.946760 24760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:55.946791 24758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:55.947067 24757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:55.947517 24752 server_base.cc:1034] running on GCE node
I20260926 19:37:55.948194 24752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:55.949283 24752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:55.950512 24752 hybrid_clock.cc:648] HybridClock initialized: now 1790451475950451 us; error 49 us; skew 500 ppm
I20260926 19:37:55.950840 24752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:37:55.953112 24752 webserver.cc:460] Webserver started at http://127.6.215.193:39581/ using document root <none> and password file <none>
I20260926 19:37:55.953809 24752 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:55.953940 24752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:55.954267 24752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:55.957228 24752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "a4c837097fbf4a77bcc2da667663919d"
format_stamp: "Formatted at 2026-09-26 19:37:55 on dist-test-slave-phbh"
server_key: "e97486ac726df785d4c54e12fabffea2"
server_key_iv: "767fb001bf35efbfe49d5d31bbc5a622"
server_key_version: "encryptionkey@0"
I20260926 19:37:55.958040 24752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "a4c837097fbf4a77bcc2da667663919d"
format_stamp: "Formatted at 2026-09-26 19:37:55 on dist-test-slave-phbh"
server_key: "e97486ac726df785d4c54e12fabffea2"
server_key_iv: "767fb001bf35efbfe49d5d31bbc5a622"
server_key_version: "encryptionkey@0"
I20260926 19:37:55.963117 24752 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260926 19:37:55.966786 24766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:55.968300 24752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 19:37:55.968439 24752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "a4c837097fbf4a77bcc2da667663919d"
format_stamp: "Formatted at 2026-09-26 19:37:55 on dist-test-slave-phbh"
server_key: "e97486ac726df785d4c54e12fabffea2"
server_key_iv: "767fb001bf35efbfe49d5d31bbc5a622"
server_key_version: "encryptionkey@0"
I20260926 19:37:55.968629 24752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:56.005009 24752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:56.005831 24752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:56.006090 24752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:56.007081 24752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:56.008821 24752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:56.008929 24752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 19:37:56.009087 24752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:56.009186 24752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:56.043602 24752 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:40413
I20260926 19:37:56.043691 24878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:40413 every 8 connection(s)
I20260926 19:37:56.044926 24752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:37:56.048094 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 24752
I20260926 19:37:56.048298 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:37:56.048929 7007 external_mini_cluster.cc:1428] Setting key c35eac865847ddaffeef6438d095d488
I20260926 19:37:56.051546 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42645
--builtin_ntp_servers=127.6.215.212:37849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 19:37:56.058059 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.047502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:54183 (local address 127.6.215.254:42645)
0926 19:37:56.047935 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:56.047940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:56.049451 (+ 1511us) server_negotiation.cc:407] Connection header received
0926 19:37:56.051505 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:56.051511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:56.051593 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:56.051735 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:56.052935 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.053630 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.055188 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.055596 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.055725 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:56.057178 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:56.057183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:56.057187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:56.057206 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:56.057246 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:56.057257 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:56.057352 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:56.057591 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 19:37:56.057692 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":233,"threads_started":1}
I20260926 19:37:56.059043 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.048125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42645 (local address 127.6.215.193:54183)
0926 19:37:56.049299 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:56.049337 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:56.051244 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:56.052065 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:56.052073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:56.052161 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:56.052724 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:56.052740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.053773 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.053778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:56.055039 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:56.055051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.055828 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.055835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:56.056072 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:56.056858 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:37:56.056861 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:56.056903 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:37:56.056987 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:56.057361 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:56.057373 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:56.057381 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 19:37:56.057560 (+ 179us) client_negotiation.cc:240] Negotiation successful
0926 19:37:56.057948 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":305,"threads_started":1}
I20260926 19:37:56.060613 24879 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42645
I20260926 19:37:56.060989 24879 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:56.061899 24879 heartbeater.cc:511] Master 127.6.215.254:42645 requested a full tablet report, sending...
I20260926 19:37:56.064311 24693 ts_manager.cc:194] Registered new tserver with Master: a4c837097fbf4a77bcc2da667663919d (127.6.215.193:40413)
I20260926 19:37:56.066584 24693 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.193:54183
W20260926 19:37:56.376825 24883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:56.377103 24883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:56.377223 24883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:56.384341 24883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:56.384442 24883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:56.384546 24883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:37:56.393671 24883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:56.394889 24883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:56.396629 24883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:56.409943 24891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:56.410014 24888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:56.410032 24889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:56.410543 24883 server_base.cc:1034] running on GCE node
I20260926 19:37:56.411141 24883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:56.412107 24883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:56.413331 24883 hybrid_clock.cc:648] HybridClock initialized: now 1790451476413283 us; error 61 us; skew 500 ppm
I20260926 19:37:56.413645 24883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:37:56.415740 24883 webserver.cc:460] Webserver started at http://127.6.215.194:43363/ using document root <none> and password file <none>
I20260926 19:37:56.416425 24883 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:56.416556 24883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:56.416958 24883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:56.419723 24883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "00472c61d86047c4911d3fe30b88d818"
format_stamp: "Formatted at 2026-09-26 19:37:56 on dist-test-slave-phbh"
server_key: "d0017e52a02fce132a45ebca0e273cf3"
server_key_iv: "cdcddafb47870c1a1aef7d410573edaa"
server_key_version: "encryptionkey@0"
I20260926 19:37:56.420526 24883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "00472c61d86047c4911d3fe30b88d818"
format_stamp: "Formatted at 2026-09-26 19:37:56 on dist-test-slave-phbh"
server_key: "d0017e52a02fce132a45ebca0e273cf3"
server_key_iv: "cdcddafb47870c1a1aef7d410573edaa"
server_key_version: "encryptionkey@0"
I20260926 19:37:56.425701 24883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 19:37:56.429416 24897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:56.430796 24883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:37:56.430976 24883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "00472c61d86047c4911d3fe30b88d818"
format_stamp: "Formatted at 2026-09-26 19:37:56 on dist-test-slave-phbh"
server_key: "d0017e52a02fce132a45ebca0e273cf3"
server_key_iv: "cdcddafb47870c1a1aef7d410573edaa"
server_key_version: "encryptionkey@0"
I20260926 19:37:56.431236 24883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:56.451880 24883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:56.452726 24883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:56.453126 24883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:56.454178 24883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:56.455709 24883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:56.455796 24883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:56.455873 24883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:56.455927 24883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:56.490069 24883 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:37925
I20260926 19:37:56.490242 25009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:37925 every 8 connection(s)
I20260926 19:37:56.491748 24883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:37:56.498605 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 24883
I20260926 19:37:56.498790 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:37:56.499343 7007 external_mini_cluster.cc:1428] Setting key fa2b54788a05e439006fc1e0240d16d9
I20260926 19:37:56.502381 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42645
--builtin_ntp_servers=127.6.215.212:37849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 19:37:56.507258 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.494550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:48405 (local address 127.6.215.254:42645)
0926 19:37:56.494741 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:56.494745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:56.496370 (+ 1625us) server_negotiation.cc:407] Connection header received
0926 19:37:56.498299 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:56.498306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:56.498391 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:56.498509 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:56.499957 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.501302 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.503780 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.504478 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.504610 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:56.506329 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:56.506336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:56.506342 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:56.506364 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:56.506407 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:56.506423 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:56.506525 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:56.506796 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 19:37:56.506910 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:37:56.508744 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.495201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42645 (local address 127.6.215.194:48405)
0926 19:37:56.496193 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:56.496237 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:56.497975 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:56.498772 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:56.498788 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:56.498918 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:56.499715 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:56.499740 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.501487 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.501493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:56.503612 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:56.503632 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.504727 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.504737 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:56.505089 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:56.505952 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:37:56.505958 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:56.506010 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:37:56.506121 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:56.506540 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:56.506554 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:56.506571 (+ 17us) client_negotiation.cc:769] Sending connection context
0926 19:37:56.506760 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 19:37:56.507187 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":316,"threads_started":1}
I20260926 19:37:56.510277 25010 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42645
I20260926 19:37:56.510623 25010 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:56.511662 25010 heartbeater.cc:511] Master 127.6.215.254:42645 requested a full tablet report, sending...
I20260926 19:37:56.513366 24693 ts_manager.cc:194] Registered new tserver with Master: 00472c61d86047c4911d3fe30b88d818 (127.6.215.194:37925)
I20260926 19:37:56.514247 24693 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.194:48405
W20260926 19:37:56.828070 25013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:56.828382 25013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:56.828536 25013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:56.836092 25013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:56.836212 25013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:56.836385 25013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:37:56.845688 25013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:56.846961 25013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:56.848706 25013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:56.861873 25018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:56.861909 25019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:56.861963 25021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:56.862799 25013 server_base.cc:1034] running on GCE node
I20260926 19:37:56.863524 25013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:56.864526 25013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:56.865792 25013 hybrid_clock.cc:648] HybridClock initialized: now 1790451476865720 us; error 75 us; skew 500 ppm
I20260926 19:37:56.866192 25013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:37:56.868635 25013 webserver.cc:460] Webserver started at http://127.6.215.195:43175/ using document root <none> and password file <none>
I20260926 19:37:56.869395 25013 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:56.869530 25013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:56.869863 25013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:56.872747 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "5e966136f79840e4828c428935da24e6"
format_stamp: "Formatted at 2026-09-26 19:37:56 on dist-test-slave-phbh"
server_key: "4cb5effc9fd8bc365289c73f10312060"
server_key_iv: "ffcbccfe088ef092b545efda5d3eaabc"
server_key_version: "encryptionkey@0"
I20260926 19:37:56.873621 25013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "5e966136f79840e4828c428935da24e6"
format_stamp: "Formatted at 2026-09-26 19:37:56 on dist-test-slave-phbh"
server_key: "4cb5effc9fd8bc365289c73f10312060"
server_key_iv: "ffcbccfe088ef092b545efda5d3eaabc"
server_key_version: "encryptionkey@0"
I20260926 19:37:56.878763 25013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 19:37:56.882287 25027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:56.883667 25013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 19:37:56.883843 25013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "5e966136f79840e4828c428935da24e6"
format_stamp: "Formatted at 2026-09-26 19:37:56 on dist-test-slave-phbh"
server_key: "4cb5effc9fd8bc365289c73f10312060"
server_key_iv: "ffcbccfe088ef092b545efda5d3eaabc"
server_key_version: "encryptionkey@0"
I20260926 19:37:56.884061 25013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:56.900846 25013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:56.901806 25013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:56.902153 25013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:56.903177 25013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:37:56.904716 25013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:37:56.904871 25013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 19:37:56.905012 25013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:37:56.905118 25013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:56.940472 25013 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:41159
I20260926 19:37:56.940522 25139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:41159 every 8 connection(s)
I20260926 19:37:56.941901 25013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:37:56.949239 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25013
I20260926 19:37:56.949431 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:37:56.949916 7007 external_mini_cluster.cc:1428] Setting key 669fc5d6b5f2961c78a3ed153a1b0a4a
I20260926 19:37:56.955404 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.944611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:55737 (local address 127.6.215.254:42645)
0926 19:37:56.944905 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:56.944912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:56.946301 (+ 1389us) server_negotiation.cc:407] Connection header received
0926 19:37:56.948039 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:56.948043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:56.948118 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:56.948215 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:56.949613 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.950535 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.952271 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.953134 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.953283 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:56.954699 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:56.954704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:56.954707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:56.954725 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:56.954757 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:56.954769 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:56.954845 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:56.955071 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 19:37:56.955149 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260926 19:37:56.956439 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.945273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42645 (local address 127.6.215.195:55737)
0926 19:37:56.946131 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:56.946166 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:56.947722 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:56.948469 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:56.948477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:56.948562 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:37:56.949379 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:56.949405 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.950695 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.950699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:56.952079 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:56.952091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.953394 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.953405 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:56.953706 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:56.954387 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:37:56.954389 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:37:56.954429 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:37:56.954507 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:37:56.954859 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:37:56.954867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:37:56.954873 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 19:37:56.955026 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 19:37:56.955335 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":263,"threads_started":1}
I20260926 19:37:56.957800 25140 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42645
I20260926 19:37:56.958099 25140 heartbeater.cc:464] Registering TS with master...
I20260926 19:37:56.958839 25140 heartbeater.cc:511] Master 127.6.215.254:42645 requested a full tablet report, sending...
I20260926 19:37:56.960201 24693 ts_manager.cc:194] Registered new tserver with Master: 5e966136f79840e4828c428935da24e6 (127.6.215.195:41159)
I20260926 19:37:56.961009 24693 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.195:55737
I20260926 19:37:56.965416 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:37:56.975131 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.969421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.6.215.254:42645)
0926 19:37:56.969614 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:56.969619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:56.970071 (+ 452us) server_negotiation.cc:407] Connection header received
0926 19:37:56.970200 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:56.970203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:56.970275 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:56.970356 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:56.970811 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.971500 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.973029 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:56.973521 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:56.973619 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:56.974563 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:56.974567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:56.974570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:56.974589 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:56.974622 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:56.974633 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:56.974701 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:56.974817 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 19:37:56.974886 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 19:37:56.978899 24693 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:37:56.985826 24693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:37:57.017076 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.999705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.6.215.193:40413)
0926 19:37:57.000331 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.000343 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.009496 (+ 9153us) server_negotiation.cc:407] Connection header received
0926 19:37:57.009603 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.009610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.009830 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.010021 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:57.010971 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.011943 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.014345 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.015793 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.015933 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.016197 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.016507 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.016700 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":292,"threads_started":1}
I20260926 19:37:57.017323 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.998113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37925 (local address 127.0.0.1:56160)
0926 19:37:56.998992 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:56.999041 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:56.999249 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:57.008529 (+ 9280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:57.008537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:57.008579 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:57.009157 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.009173 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.012039 (+ 2866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.012047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.014723 (+ 2676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.014737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.016404 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.016411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.016577 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.016610 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 19:37:57.016713 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 19:37:57.016938 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":164,"thread_start_us":4240,"threads_started":1}
I20260926 19:37:57.017663 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.998449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56160 (local address 127.6.215.194:37925)
0926 19:37:57.007803 (+ 9354us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.007814 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.007850 (+ 36us) server_negotiation.cc:407] Connection header received
0926 19:37:57.007954 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.007963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.008251 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.009404 (+ 1153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:57.009704 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.011891 (+ 2187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.014952 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.016203 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.016355 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.016613 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.016746 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.016917 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9171,"thread_start_us":9024,"threads_started":1}
I20260926 19:37:57.017863 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.998863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33754)
0926 19:37:57.009359 (+ 10496us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.009375 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:57.009484 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:57.010255 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:57.010286 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:57.010324 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:57.010766 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.010778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.012121 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.012126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.014159 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.014171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.015992 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.015998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.016181 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.016242 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 19:37:57.017318 (+ 1076us) client_negotiation.cc:240] Negotiation successful
0926 19:37:57.017448 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10371,"mutex_wait_us":237,"thread_start_us":9467,"threads_started":1}
I20260926 19:37:57.018230 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.998741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:41159 (local address 127.0.0.1:35298)
0926 19:37:57.008634 (+ 9893us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.008660 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:57.008814 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:57.009960 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:57.009964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:57.010006 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:57.010493 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.010506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.012354 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.012360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.015218 (+ 2858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.015232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.017331 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.017340 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.017544 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.017588 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 19:37:57.017675 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 19:37:57.017835 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9751,"mutex_wait_us":133,"thread_start_us":10325,"threads_started":1}
I20260926 19:37:57.018452 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:56.999306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35298 (local address 127.6.215.195:41159)
0926 19:37:57.008167 (+ 8861us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.008174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.008803 (+ 629us) server_negotiation.cc:407] Connection header received
0926 19:37:57.008993 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.009005 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.009606 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.009829 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:57.010725 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.012218 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.015393 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.016842 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.016957 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.017190 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.017951 (+ 761us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.018107 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8677,"thread_start_us":8534,"threads_started":1}
I20260926 19:37:57.021740 24814 tablet_service.cc:1467] Processing CreateTablet for tablet 628daa916f634f998fd4ced98094e043 (DEFAULT_TABLE table=test-table [id=acb8a8d621534f1388e0f802d6b3614e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:57.021740 24945 tablet_service.cc:1467] Processing CreateTablet for tablet 628daa916f634f998fd4ced98094e043 (DEFAULT_TABLE table=test-table [id=acb8a8d621534f1388e0f802d6b3614e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:57.022153 25075 tablet_service.cc:1467] Processing CreateTablet for tablet 628daa916f634f998fd4ced98094e043 (DEFAULT_TABLE table=test-table [id=acb8a8d621534f1388e0f802d6b3614e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:37:57.023180 24814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628daa916f634f998fd4ced98094e043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:57.023175 24945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628daa916f634f998fd4ced98094e043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:57.023327 25075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628daa916f634f998fd4ced98094e043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:57.035162 25158 tablet_bootstrap.cc:492] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Bootstrap starting.
I20260926 19:37:57.037577 25159 tablet_bootstrap.cc:492] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: Bootstrap starting.
I20260926 19:37:57.038394 25160 tablet_bootstrap.cc:492] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Bootstrap starting.
I20260926 19:37:57.039513 25158 tablet_bootstrap.cc:654] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:57.040849 25158 log.cc:824] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:57.042068 25160 tablet_bootstrap.cc:654] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:57.043684 25160 log.cc:824] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:57.044133 25159 tablet_bootstrap.cc:654] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:57.045421 25159 log.cc:824] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: Log is configured to *not* fsync() on all Append() calls
I20260926 19:37:57.051278 25160 tablet_bootstrap.cc:492] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: No bootstrap required, opened a new log
I20260926 19:37:57.051538 25159 tablet_bootstrap.cc:492] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: No bootstrap required, opened a new log
I20260926 19:37:57.051663 25160 ts_tablet_manager.cc:1397] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260926 19:37:57.051709 25158 tablet_bootstrap.cc:492] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: No bootstrap required, opened a new log
I20260926 19:37:57.051844 25159 ts_tablet_manager.cc:1397] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260926 19:37:57.051954 25158 ts_tablet_manager.cc:1397] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260926 19:37:57.062418 25159 raft_consensus.cc:348] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.062832 25159 raft_consensus.cc:374] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:57.062618 25158 raft_consensus.cc:348] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.062618 25160 raft_consensus.cc:348] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.062942 25159 raft_consensus.cc:729] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c837097fbf4a77bcc2da667663919d, State: Initialized, Role: FOLLOWER
I20260926 19:37:57.063091 25158 raft_consensus.cc:374] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:57.063081 25160 raft_consensus.cc:374] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:57.063189 25160 raft_consensus.cc:729] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00472c61d86047c4911d3fe30b88d818, State: Initialized, Role: FOLLOWER
I20260926 19:37:57.063189 25158 raft_consensus.cc:729] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e966136f79840e4828c428935da24e6, State: Initialized, Role: FOLLOWER
I20260926 19:37:57.063651 25159 consensus_queue.cc:260] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.063769 25158 consensus_queue.cc:260] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.063876 25160 consensus_queue.cc:260] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.065461 25159 ts_tablet_manager.cc:1428] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260926 19:37:57.065784 24879 heartbeater.cc:503] Master 127.6.215.254:42645 was elected leader, sending a full tablet report...
I20260926 19:37:57.070101 25140 heartbeater.cc:503] Master 127.6.215.254:42645 was elected leader, sending a full tablet report...
I20260926 19:37:57.070536 25010 heartbeater.cc:503] Master 127.6.215.254:42645 was elected leader, sending a full tablet report...
I20260926 19:37:57.070847 25158 ts_tablet_manager.cc:1428] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Time spent starting tablet: real 0.019s user 0.015s sys 0.002s
I20260926 19:37:57.072372 25160 ts_tablet_manager.cc:1428] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Time spent starting tablet: real 0.020s user 0.017s sys 0.001s
I20260926 19:37:57.125598 25165 raft_consensus.cc:482] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:37:57.125952 25165 raft_consensus.cc:504] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.127882 25165 leader_election.cc:288] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00472c61d86047c4911d3fe30b88d818 (127.6.215.194:37925), a4c837097fbf4a77bcc2da667663919d (127.6.215.193:40413)
I20260926 19:37:57.134268 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.128201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54613 (local address 127.6.215.194:37925)
0926 19:37:57.128397 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.128404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.128426 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:37:57.128587 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.128595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.128676 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.128790 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:57.129476 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.130415 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.132313 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.133363 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.133475 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.133523 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.133917 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.134006 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:37:57.134390 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.128088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37925 (local address 127.6.215.195:54613)
0926 19:37:57.128281 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.128302 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:57.128423 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:57.128952 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:57.128956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:57.128988 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:57.129237 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.129248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.130568 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.130574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.132149 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.132164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.133538 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.133545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.133743 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.133781 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 19:37:57.133886 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 19:37:57.134025 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":3}
I20260926 19:37:57.135079 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.128288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.6.215.195:56781)
0926 19:37:57.129009 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.129022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 19:37:57.129130 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:37:57.129473 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:37:57.129478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:37:57.129504 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:37:57.129882 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.129894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.131034 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.131040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.132849 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:37:57.132862 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.134316 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.134325 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:37:57.134548 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.134579 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:37:57.134704 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 19:37:57.134815 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":414,"threads_started":1}
I20260926 19:37:57.135264 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.128623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:56781 (local address 127.6.215.193:40413)
0926 19:37:57.128866 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.128872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.129137 (+ 265us) server_negotiation.cc:407] Connection header received
0926 19:37:57.129220 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.129225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.129316 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.129415 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:37:57.130037 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.130882 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.133075 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.134140 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.134285 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.134345 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.134745 (+ 400us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.134898 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260926 19:37:57.135625 24965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628daa916f634f998fd4ced98094e043" candidate_uuid: "5e966136f79840e4828c428935da24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00472c61d86047c4911d3fe30b88d818" is_pre_election: true
I20260926 19:37:57.136205 24965 raft_consensus.cc:2454] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e966136f79840e4828c428935da24e6 in term 0.
I20260926 19:37:57.136201 24834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628daa916f634f998fd4ced98094e043" candidate_uuid: "5e966136f79840e4828c428935da24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4c837097fbf4a77bcc2da667663919d" is_pre_election: true
I20260926 19:37:57.136811 24834 raft_consensus.cc:2454] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e966136f79840e4828c428935da24e6 in term 0.
I20260926 19:37:57.136878 25029 leader_election.cc:302] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00472c61d86047c4911d3fe30b88d818, 5e966136f79840e4828c428935da24e6; no voters:
I20260926 19:37:57.137261 25165 raft_consensus.cc:2781] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:37:57.137413 25165 raft_consensus.cc:482] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:37:57.137529 25165 raft_consensus.cc:3037] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:57.139642 25165 raft_consensus.cc:504] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.140308 25165 leader_election.cc:288] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [CANDIDATE]: Term 1 election: Requested vote from peers 00472c61d86047c4911d3fe30b88d818 (127.6.215.194:37925), a4c837097fbf4a77bcc2da667663919d (127.6.215.193:40413)
I20260926 19:37:57.140662 24965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628daa916f634f998fd4ced98094e043" candidate_uuid: "5e966136f79840e4828c428935da24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00472c61d86047c4911d3fe30b88d818"
I20260926 19:37:57.140877 24965 raft_consensus.cc:3037] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:57.140844 24834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628daa916f634f998fd4ced98094e043" candidate_uuid: "5e966136f79840e4828c428935da24e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4c837097fbf4a77bcc2da667663919d"
I20260926 19:37:57.141109 24834 raft_consensus.cc:3037] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:57.143229 24965 raft_consensus.cc:2454] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e966136f79840e4828c428935da24e6 in term 1.
I20260926 19:37:57.143277 24834 raft_consensus.cc:2454] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e966136f79840e4828c428935da24e6 in term 1.
I20260926 19:37:57.143610 25029 leader_election.cc:302] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00472c61d86047c4911d3fe30b88d818, 5e966136f79840e4828c428935da24e6; no voters:
I20260926 19:37:57.143916 25165 raft_consensus.cc:2781] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:37:57.144402 25165 raft_consensus.cc:686] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5e966136f79840e4828c428935da24e6, State: Running, Role: LEADER
I20260926 19:37:57.144944 25165 consensus_queue.cc:237] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } }
I20260926 19:37:57.149399 24693 catalog_manager.cc:5495] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e966136f79840e4828c428935da24e6 (127.6.215.195). New cstate: current_term: 1 leader_uuid: "5e966136f79840e4828c428935da24e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e966136f79840e4828c428935da24e6" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 } health_report { overall_health: UNKNOWN } } }
I20260926 19:37:57.167474 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.162141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.6.215.195:41159)
0926 19:37:57.162332 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.162337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.162462 (+ 125us) server_negotiation.cc:407] Connection header received
0926 19:37:57.162658 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.162661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.162738 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.162864 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:37:57.163226 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.163936 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.165441 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.165845 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.165938 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.166420 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:37:57.166898 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:37:57.167008 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.167187 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.167249 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 19:37:57.175808 24965 raft_consensus.cc:1261] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 1 FOLLOWER]: Refusing update from remote peer 5e966136f79840e4828c428935da24e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:37:57.175808 24834 raft_consensus.cc:1261] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 1 FOLLOWER]: Refusing update from remote peer 5e966136f79840e4828c428935da24e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:37:57.176887 25165 consensus_queue.cc:1035] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00472c61d86047c4911d3fe30b88d818" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:37:57.177206 25168 consensus_queue.cc:1035] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4c837097fbf4a77bcc2da667663919d" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 40413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:37:57.187150 25171 mvcc.cc:205] Tried to move back new op lower bound from 7333689250501644288 to 7333689250389602304. Current Snapshot: MvccSnapshot[applied={T|T < 7333689250501644288}]
I20260926 19:37:57.189502 25172 mvcc.cc:205] Tried to move back new op lower bound from 7333689250501644288 to 7333689250389602304. Current Snapshot: MvccSnapshot[applied={T|T < 7333689250501644288}]
I20260926 19:37:57.203814 24692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:37:57.204061 24692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:37:57.207966 24692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76b1425c5f334e2eb965429014f66039: Sending DeleteTablet for 3 replicas of tablet 628daa916f634f998fd4ced98094e043
I20260926 19:37:57.209129 25075 tablet_service.cc:1514] Processing DeleteTablet for tablet 628daa916f634f998fd4ced98094e043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:57 UTC) from {username='slave'} at 127.0.0.1:35298
I20260926 19:37:57.209210 24814 tablet_service.cc:1514] Processing DeleteTablet for tablet 628daa916f634f998fd4ced98094e043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:57 UTC) from {username='slave'} at 127.0.0.1:33754
I20260926 19:37:57.209873 24945 tablet_service.cc:1514] Processing DeleteTablet for tablet 628daa916f634f998fd4ced98094e043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:37:57 UTC) from {username='slave'} at 127.0.0.1:56160
I20260926 19:37:57.210681 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 24752
I20260926 19:37:57.213903 25181 tablet_replica.cc:330] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: stopping tablet replica
I20260926 19:37:57.214501 25183 tablet_replica.cc:330] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: stopping tablet replica
I20260926 19:37:57.215085 25181 raft_consensus.cc:2229] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:37:57.215659 25183 raft_consensus.cc:2229] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:37:57.216274 25183 raft_consensus.cc:2258] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:57.217101 25181 raft_consensus.cc:2258] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:57.220041 25183 ts_tablet_manager.cc:1905] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:57.220860 25181 ts_tablet_manager.cc:1905] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:57.225418 25182 tablet_replica.cc:330] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: stopping tablet replica
I20260926 19:37:57.225867 25182 raft_consensus.cc:2229] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:37:57.226387 25182 raft_consensus.cc:2258] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:37:57.230669 25182 ts_tablet_manager.cc:1905] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:37:57.231832 25181 ts_tablet_manager.cc:1918] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:57.232031 25181 log.cc:1196] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/628daa916f634f998fd4ced98094e043
I20260926 19:37:57.232615 25181 ts_tablet_manager.cc:1939] T 628daa916f634f998fd4ced98094e043 P 5e966136f79840e4828c428935da24e6: Deleting consensus metadata
I20260926 19:37:57.233939 25183 ts_tablet_manager.cc:1918] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:57.234630 24677 catalog_manager.cc:4841] TS 5e966136f79840e4828c428935da24e6 (127.6.215.195:41159): tablet 628daa916f634f998fd4ced98094e043 (table test-table [id=acb8a8d621534f1388e0f802d6b3614e]) successfully deleted
I20260926 19:37:57.235157 25183 log.cc:1196] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/628daa916f634f998fd4ced98094e043
I20260926 19:37:57.236063 25183 ts_tablet_manager.cc:1939] T 628daa916f634f998fd4ced98094e043 P 00472c61d86047c4911d3fe30b88d818: Deleting consensus metadata
I20260926 19:37:57.237895 24678 catalog_manager.cc:4841] TS 00472c61d86047c4911d3fe30b88d818 (127.6.215.194:37925): tablet 628daa916f634f998fd4ced98094e043 (table test-table [id=acb8a8d621534f1388e0f802d6b3614e]) successfully deleted
I20260926 19:37:57.238447 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.211753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.6.215.193:40413)
0926 19:37:57.211952 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.211960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.229384 (+ 17424us) server_negotiation.cc:407] Connection header received
0926 19:37:57.229683 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.229690 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.229803 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.229888 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:57.230578 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.231792 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.233958 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.234715 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.234814 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.236439 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:57.236451 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:57.236470 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:57.237320 (+ 850us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:57.237393 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:57.237419 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:57.237513 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.237919 (+ 406us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.238035 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:37:57.239128 24874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:57.345134 25182 ts_tablet_manager.cc:1918] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:37:57.345420 25182 log.cc:1196] T 628daa916f634f998fd4ced98094e043 P a4c837097fbf4a77bcc2da667663919d: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/628daa916f634f998fd4ced98094e043
I20260926 19:37:57.345845 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 24752
I20260926 19:37:57.363251 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 24883
W20260926 19:37:57.363317 24679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.215.193:40413 (error 108)
I20260926 19:37:57.365445 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.364961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33780)
0926 19:37:57.365158 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.365299 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 19:37:57.365775 24679 catalog_manager.cc:4584] TS a4c837097fbf4a77bcc2da667663919d (127.6.215.193:40413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 628daa916f634f998fd4ced98094e043: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
I20260926 19:37:57.371796 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.364541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.6.215.194:37925)
0926 19:37:57.364724 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.364728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.364744 (+ 16us) server_negotiation.cc:407] Connection header received
0926 19:37:57.364849 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.364854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.364946 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.365069 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:57.365682 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.366866 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.368212 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.368989 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.369132 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.370199 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:57.370213 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:57.370232 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:57.371005 (+ 773us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:57.371076 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:57.371102 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:57.371216 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.371335 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.371447 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 19:37:57.372555 25005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:57.419486 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.418911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33790)
0926 19:37:57.419166 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.419294 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 19:37:57.420693 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.420263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33802)
0926 19:37:57.420440 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.420532 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 19:37:57.468577 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 24883
I20260926 19:37:57.481720 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.481110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33806)
0926 19:37:57.481372 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.481491 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 19:37:57.483002 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.482533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33818)
0926 19:37:57.482726 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.482824 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 19:37:57.486837 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25013
I20260926 19:37:57.496017 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.487901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35306 (local address 127.6.215.195:41159)
0926 19:37:57.488062 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:57.488068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:57.488087 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:37:57.488225 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:57.488230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:57.488335 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:57.488445 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:57.488953 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.490127 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.492105 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:57.492970 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:57.493107 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:57.494371 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:57.494382 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:57.494402 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:37:57.495192 (+ 790us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:57.495248 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:37:57.495271 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:57.495374 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:57.495561 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 19:37:57.495660 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 19:37:57.496744 25135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:57.563898 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.563266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33824)
0926 19:37:57.563551 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.563709 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260926 19:37:57.565137 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.564679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33826)
0926 19:37:57.564919 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.565006 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260926 19:37:57.621918 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25013
I20260926 19:37:57.642621 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 24662
I20260926 19:37:57.643767 24723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:37:57.753099 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:57.752231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:40413 (local address 127.0.0.1:33836)
0926 19:37:57.752425 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 19:37:57.752578 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 19:37:57.753350 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 24662
2026-09-26T19:37:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2775 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:37:57 dist-test-slave-phbh krb5kdc[25192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:37:57 dist-test-slave-phbh krb5kdc[25192](info): set up 2 sockets
Sep 26 19:37:57 dist-test-slave-phbh krb5kdc[25192](info): commencing operation
krb5kdc: starting...
W20260926 19:37:58.896981 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:37:59 dist-test-slave-phbh krb5kdc[25192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:37:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:37:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:37:59.151834 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42769
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:37:59.479743 25205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:37:59.480036 25205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:37:59.480213 25205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:37:59.487530 25205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:37:59.487640 25205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:37:59.487746 25205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:37:59.487852 25205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:37:59.487929 25205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:37:59.498112 25205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:37:59.499440 25205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:37:59.500751 25205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:37:59.512333 25210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:59.512486 25211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:37:59.512454 25213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:37:59.513449 25205 server_base.cc:1034] running on GCE node
I20260926 19:37:59.514487 25205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:37:59.515904 25205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:37:59.517127 25205 hybrid_clock.cc:648] HybridClock initialized: now 1790451479517048 us; error 49 us; skew 500 ppm
I20260926 19:37:59.517489 25205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:37:59 dist-test-slave-phbh krb5kdc[25192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:37:59.520447 25205 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:37:59.521903 25205 webserver.cc:460] Webserver started at http://127.6.215.254:34959/ using document root <none> and password file <none>
I20260926 19:37:59.522552 25205 fs_manager.cc:359] Metadata directory not provided
I20260926 19:37:59.522653 25205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:37:59.522886 25205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:37:59.525861 25205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "69962ee3c6624772958b402d224462e7"
format_stamp: "Formatted at 2026-09-26 19:37:59 on dist-test-slave-phbh"
server_key: "a74db83a9527f755ff38a4004aff84d9"
server_key_iv: "da7cd7b85798cae900be90d8f305e191"
server_key_version: "encryptionkey@0"
I20260926 19:37:59.526543 25205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "69962ee3c6624772958b402d224462e7"
format_stamp: "Formatted at 2026-09-26 19:37:59 on dist-test-slave-phbh"
server_key: "a74db83a9527f755ff38a4004aff84d9"
server_key_iv: "da7cd7b85798cae900be90d8f305e191"
server_key_version: "encryptionkey@0"
I20260926 19:37:59.531584 25205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:37:59.534832 25220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:37:59.536098 25205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:37:59.536280 25205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "69962ee3c6624772958b402d224462e7"
format_stamp: "Formatted at 2026-09-26 19:37:59 on dist-test-slave-phbh"
server_key: "a74db83a9527f755ff38a4004aff84d9"
server_key_iv: "da7cd7b85798cae900be90d8f305e191"
server_key_version: "encryptionkey@0"
I20260926 19:37:59.536464 25205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:37:59.553874 25205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:37:59.558180 25205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:37:59.558494 25205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:37:59.577291 25205 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42769
I20260926 19:37:59.577299 25271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42769 every 8 connection(s)
I20260926 19:37:59.578640 25205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:37:59.582404 25272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:37:59.585220 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25205
I20260926 19:37:59.585407 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:37:59.585970 7007 external_mini_cluster.cc:1428] Setting key 8d679210bf0ddd7fd5128e2a60d5aef3
I20260926 19:37:59.595197 25272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7: Bootstrap starting.
I20260926 19:37:59.599140 25272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7: Neither blocks nor log segments found. Creating new log.
I20260926 19:37:59.600507 25272 log.cc:824] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7: Log is configured to *not* fsync() on all Append() calls
Sep 26 19:37:59 dist-test-slave-phbh krb5kdc[25192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:37:59.603935 25272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7: No bootstrap required, opened a new log
I20260926 19:37:59.610972 25275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:37:59.588066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.6.215.254:42769)
0926 19:37:59.589030 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0926 19:37:59.589042 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:37:59.589084 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:37:59.590767 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:37:59.590794 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:37:59.591200 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:37:59.591553 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:37:59.593487 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:59.594821 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:59.596744 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:37:59.597501 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:37:59.597652 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:37:59.603119 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:37:59.603127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:37:59.603143 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:37:59.603179 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:37:59.606505 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:59.607463 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:59.607472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:59.607482 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:59.607675 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:37:59.608117 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:37:59.608120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:37:59.608122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:37:59.608671 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:37:59.608968 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0926 19:37:59.609464 (+ 496us) server_negotiation.cc:299] Negotiation successful
0926 19:37:59.609846 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":513,"threads_started":1}
I20260926 19:37:59.613018 25272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69962ee3c6624772958b402d224462e7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42769 } }
I20260926 19:37:59.613402 25272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:37:59.613531 25272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69962ee3c6624772958b402d224462e7, State: Initialized, Role: FOLLOWER
I20260926 19:37:59.614156 25272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69962ee3c6624772958b402d224462e7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42769 } }
I20260926 19:37:59.614367 25272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:37:59.614459 25272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:37:59.614574 25272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:37:59.616669 25272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69962ee3c6624772958b402d224462e7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42769 } }
I20260926 19:37:59.617180 25272 leader_election.cc:302] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69962ee3c6624772958b402d224462e7; no voters:
I20260926 19:37:59.617841 25272 leader_election.cc:288] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:37:59.617978 25277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:37:59.618743 25277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [term 1 LEADER]: Becoming Leader. State: Replica: 69962ee3c6624772958b402d224462e7, State: Running, Role: LEADER
I20260926 19:37:59.619325 25277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69962ee3c6624772958b402d224462e7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42769 } }
I20260926 19:37:59.619952 25272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:37:59.622164 25279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69962ee3c6624772958b402d224462e7. Latest consensus state: current_term: 1 leader_uuid: "69962ee3c6624772958b402d224462e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69962ee3c6624772958b402d224462e7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42769 } } }
I20260926 19:37:59.622622 25279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:59.623201 25278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69962ee3c6624772958b402d224462e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69962ee3c6624772958b402d224462e7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42769 } } }
I20260926 19:37:59.623754 25278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7 [sys.catalog]: This master's current role is: LEADER
I20260926 19:37:59.625320 25285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:37:59.630766 25285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:37:59.639632 25285 catalog_manager.cc:1347] Generated new cluster ID: cec2723a0c354cea916dfda4958d09b6
I20260926 19:37:59.639750 25285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:37:59.649771 25285 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:37:59.651553 25285 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:37:59.669927 25285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7: Generated new TSK 0
I20260926 19:37:59.671048 25285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:37:59.732573 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42769
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:00.065086 25298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:00.065364 25298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:00.065557 25298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:00.078132 25298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:00.078305 25298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:00.078545 25298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:38:00.094398 25298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:00.095834 25298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:00.097522 25298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:00.113073 25304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:00.113152 25303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:00.114255 25298 server_base.cc:1034] running on GCE node
W20260926 19:38:00.114290 25306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:00.115025 25298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:00.116180 25298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:00.120869 25298 hybrid_clock.cc:648] HybridClock initialized: now 1790451480120800 us; error 56 us; skew 500 ppm
I20260926 19:38:00.121316 25298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:00 dist-test-slave-phbh krb5kdc[25192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:00.124615 25298 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:38:00.126215 25298 webserver.cc:460] Webserver started at http://127.6.215.193:40941/ using document root <none> and password file <none>
I20260926 19:38:00.126891 25298 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:00.127034 25298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:00.127373 25298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:00.130332 25298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "9e78d35f76ba4868aca5c55ff3d548c0"
format_stamp: "Formatted at 2026-09-26 19:38:00 on dist-test-slave-phbh"
server_key: "ca81d0f30b7d51e3d6c6b024910f5898"
server_key_iv: "01df0d50ef370e57270bb53eb69a88ed"
server_key_version: "encryptionkey@0"
I20260926 19:38:00.131122 25298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "9e78d35f76ba4868aca5c55ff3d548c0"
format_stamp: "Formatted at 2026-09-26 19:38:00 on dist-test-slave-phbh"
server_key: "ca81d0f30b7d51e3d6c6b024910f5898"
server_key_iv: "01df0d50ef370e57270bb53eb69a88ed"
server_key_version: "encryptionkey@0"
I20260926 19:38:00.136392 25298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 19:38:00.139911 25313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:00.141369 25298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:00.141537 25298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "9e78d35f76ba4868aca5c55ff3d548c0"
format_stamp: "Formatted at 2026-09-26 19:38:00 on dist-test-slave-phbh"
server_key: "ca81d0f30b7d51e3d6c6b024910f5898"
server_key_iv: "01df0d50ef370e57270bb53eb69a88ed"
server_key_version: "encryptionkey@0"
I20260926 19:38:00.141741 25298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:00.156378 25298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:00.160636 25298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:00.160946 25298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:00.161922 25298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:00.163204 25298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:00.163275 25298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:00.163437 25298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:00.163509 25298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:00.195533 25298 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:32821
I20260926 19:38:00.195572 25425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:32821 every 8 connection(s)
I20260926 19:38:00.196962 25298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:38:00.207237 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25298
I20260926 19:38:00.207437 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:38:00.207885 7007 external_mini_cluster.cc:1428] Setting key e0abfad921577bc9fcec9a0ebb2572b2
Sep 26 19:38:00 dist-test-slave-phbh krb5kdc[25192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:00.217007 25428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:00.199511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:38937 (local address 127.6.215.254:42769)
0926 19:38:00.199947 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:00.199951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:00.200957 (+ 1006us) server_negotiation.cc:407] Connection header received
0926 19:38:00.202827 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:00.202833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:00.202921 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:00.203047 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:00.204513 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.205316 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.206879 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.207282 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.207418 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:00.211254 (+ 3836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:00.211259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:00.211263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:00.211293 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:00.213944 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:00.214668 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:00.214675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:00.214679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:00.214800 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:00.215241 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:00.215245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:00.215249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:00.215508 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:00.215673 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:00.216367 (+ 694us) server_negotiation.cc:299] Negotiation successful
0926 19:38:00.216597 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":213,"threads_started":1}
I20260926 19:38:00.217963 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:00.200042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42769 (local address 127.6.215.193:38937)
0926 19:38:00.200733 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:00.200835 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:00.202561 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:00.203282 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:00.203290 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:00.203813 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:00.204341 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:00.204356 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.205486 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.205490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:00.206717 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:00.206728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.207530 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.207537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:00.207763 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:00.208567 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:00.208602 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:00.211030 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:00.214099 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:00.214108 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:00.214124 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:00.214546 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:00.214911 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:00.214916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:00.214920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:00.215119 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:00.215643 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:00.215651 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:00.215994 (+ 343us) client_negotiation.cc:769] Sending connection context
0926 19:38:00.216270 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 19:38:00.216829 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":262,"threads_started":1}
I20260926 19:38:00.219576 25426 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42769
I20260926 19:38:00.219914 25426 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:00.220818 25426 heartbeater.cc:511] Master 127.6.215.254:42769 requested a full tablet report, sending...
I20260926 19:38:00.223284 25237 ts_manager.cc:194] Registered new tserver with Master: 9e78d35f76ba4868aca5c55ff3d548c0 (127.6.215.193:32821)
I20260926 19:38:00.225559 25237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:38937
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:38:00.264461 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42769
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:00.601652 25432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:00.601927 25432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:00.602087 25432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:00.609615 25432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:00.609750 25432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:00.609938 25432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:38:00.619829 25432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:00.621268 25432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:00.622587 25432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:00.635859 25437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:00.635969 25440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:00.635946 25438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:00.637079 25432 server_base.cc:1034] running on GCE node
I20260926 19:38:00.637641 25432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:00.639200 25432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:00.640461 25432 hybrid_clock.cc:648] HybridClock initialized: now 1790451480640399 us; error 54 us; skew 500 ppm
I20260926 19:38:00.640843 25432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:00 dist-test-slave-phbh krb5kdc[25192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:00.653688 25432 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:38:00.655369 25432 webserver.cc:460] Webserver started at http://127.6.215.194:40613/ using document root <none> and password file <none>
I20260926 19:38:00.656123 25432 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:00.656272 25432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:00.656598 25432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:00.659556 25432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "03c2254ea726440aa1801fb249b14d5f"
format_stamp: "Formatted at 2026-09-26 19:38:00 on dist-test-slave-phbh"
server_key: "f6935c28bc1e1ec7c11f92e873cfcb07"
server_key_iv: "0e700bed2080de8527e7ce730c9a3306"
server_key_version: "encryptionkey@0"
I20260926 19:38:00.660339 25432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "03c2254ea726440aa1801fb249b14d5f"
format_stamp: "Formatted at 2026-09-26 19:38:00 on dist-test-slave-phbh"
server_key: "f6935c28bc1e1ec7c11f92e873cfcb07"
server_key_iv: "0e700bed2080de8527e7ce730c9a3306"
server_key_version: "encryptionkey@0"
I20260926 19:38:00.665452 25432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 19:38:00.668911 25447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:00.670415 25432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:00.670578 25432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "03c2254ea726440aa1801fb249b14d5f"
format_stamp: "Formatted at 2026-09-26 19:38:00 on dist-test-slave-phbh"
server_key: "f6935c28bc1e1ec7c11f92e873cfcb07"
server_key_iv: "0e700bed2080de8527e7ce730c9a3306"
server_key_version: "encryptionkey@0"
I20260926 19:38:00.670763 25432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:00.683319 25432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:00.687397 25432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:00.687697 25432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:00.688640 25432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:00.690043 25432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:00.690124 25432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:00.690284 25432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:00.690333 25432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:00.723714 25432 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:37307
I20260926 19:38:00.723765 25559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:37307 every 8 connection(s)
I20260926 19:38:00.725116 25432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:38:00.730268 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25432
I20260926 19:38:00.730455 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:38:00.730911 7007 external_mini_cluster.cc:1428] Setting key dcb97602963434edeb35b8c259e5e12d
Sep 26 19:38:00 dist-test-slave-phbh krb5kdc[25192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:00.744885 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:00.727688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:55429 (local address 127.6.215.254:42769)
0926 19:38:00.728144 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:00.728149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:00.728980 (+ 831us) server_negotiation.cc:407] Connection header received
0926 19:38:00.730894 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:00.730900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:00.730995 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:00.731127 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:00.732625 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.733416 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.734925 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.735352 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.735484 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:00.738896 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:00.738901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:00.738904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:00.738935 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:00.741754 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:00.742483 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:00.742489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:00.742492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:00.742642 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:00.743073 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:00.743078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:00.743081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:00.743351 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:00.743510 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:00.744217 (+ 707us) server_negotiation.cc:299] Negotiation successful
0926 19:38:00.744443 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":236,"threads_started":1}
I20260926 19:38:00.745934 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:00.728086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42769 (local address 127.6.215.194:55429)
0926 19:38:00.728820 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:00.728867 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:00.730627 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:00.731365 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:00.731373 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:00.731880 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:00.732442 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:00.732459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.733558 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.733570 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:00.734760 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:00.734772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:00.735568 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:00.735584 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:00.735804 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:00.736480 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:00.736502 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:00.738669 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:00.741901 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:00.741909 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:00.741926 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:00.742355 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:00.742754 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:00.742760 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:00.742764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:00.742955 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:00.743481 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:00.743489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:00.743829 (+ 340us) client_negotiation.cc:769] Sending connection context
0926 19:38:00.744118 (+ 289us) client_negotiation.cc:240] Negotiation successful
0926 19:38:00.744669 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":253,"threads_started":1}
I20260926 19:38:00.747431 25560 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42769
I20260926 19:38:00.747786 25560 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:00.748672 25560 heartbeater.cc:511] Master 127.6.215.254:42769 requested a full tablet report, sending...
I20260926 19:38:00.750288 25237 ts_manager.cc:194] Registered new tserver with Master: 03c2254ea726440aa1801fb249b14d5f (127.6.215.194:37307)
I20260926 19:38:00.751585 25237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:55429
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:38:00.787900 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42769
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:38551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:01.111142 25566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:01.111404 25566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:01.111498 25566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:01.118726 25566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:01.118837 25566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:01.118950 25566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:38:01.128890 25566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:01.130249 25566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:01.131512 25566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:01.144606 25571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:01.144572 25572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:01.144981 25574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:01.145839 25566 server_base.cc:1034] running on GCE node
I20260926 19:38:01.146450 25566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:01.147307 25566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:01.148500 25566 hybrid_clock.cc:648] HybridClock initialized: now 1790451481148430 us; error 64 us; skew 500 ppm
I20260926 19:38:01.148825 25566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:01 dist-test-slave-phbh krb5kdc[25192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:01.151911 25566 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:38:01.153453 25566 webserver.cc:460] Webserver started at http://127.6.215.195:39633/ using document root <none> and password file <none>
I20260926 19:38:01.154309 25566 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:01.154452 25566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:01.154790 25566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:01.157780 25566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "d32d6e965a5f43cda1f8ee7d37ee584f"
format_stamp: "Formatted at 2026-09-26 19:38:01 on dist-test-slave-phbh"
server_key: "71a3cf47e9abb81325ef42d7120bec4e"
server_key_iv: "c2958f5a2324348301ec73662300386c"
server_key_version: "encryptionkey@0"
I20260926 19:38:01.158555 25566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "d32d6e965a5f43cda1f8ee7d37ee584f"
format_stamp: "Formatted at 2026-09-26 19:38:01 on dist-test-slave-phbh"
server_key: "71a3cf47e9abb81325ef42d7120bec4e"
server_key_iv: "c2958f5a2324348301ec73662300386c"
server_key_version: "encryptionkey@0"
I20260926 19:38:01.163516 25566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:38:01.166815 25581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:01.168076 25566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:38:01.168238 25566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "d32d6e965a5f43cda1f8ee7d37ee584f"
format_stamp: "Formatted at 2026-09-26 19:38:01 on dist-test-slave-phbh"
server_key: "71a3cf47e9abb81325ef42d7120bec4e"
server_key_iv: "c2958f5a2324348301ec73662300386c"
server_key_version: "encryptionkey@0"
I20260926 19:38:01.168413 25566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:01.200095 25566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:01.204408 25566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:01.204705 25566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:01.205668 25566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:01.206925 25566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:01.207001 25566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:01.207185 25566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:01.207237 25566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:01.229607 25426 heartbeater.cc:503] Master 127.6.215.254:42769 was elected leader, sending a full tablet report...
I20260926 19:38:01.241659 25566 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:40993
I20260926 19:38:01.241699 25693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:40993 every 8 connection(s)
I20260926 19:38:01.242945 25566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:38:01.251560 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25566
I20260926 19:38:01.251760 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:38:01.252254 7007 external_mini_cluster.cc:1428] Setting key 5b89e56dc38192390fc568fd3821c664
Sep 26 19:38:01 dist-test-slave-phbh krb5kdc[25192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:01.263599 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.245813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54521 (local address 127.6.215.254:42769)
0926 19:38:01.246253 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.246258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.247405 (+ 1147us) server_negotiation.cc:407] Connection header received
0926 19:38:01.249348 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.249354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.249443 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.249575 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:01.251128 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.252082 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.253921 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.254507 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.254635 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.258153 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:01.258158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:01.258162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:01.258194 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:01.260860 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.261576 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.261581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.261583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.261715 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.262127 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.262131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.262133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.262398 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:01.262509 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.263172 (+ 663us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.263333 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":217,"threads_started":1}
I20260926 19:38:01.264530 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.246446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42769 (local address 127.6.215.195:54521)
0926 19:38:01.247250 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:01.247291 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:01.249073 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:01.249818 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:01.249827 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:01.250399 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:01.250957 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.250975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.252235 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.252240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.253787 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.253799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.254709 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.254714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.254985 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.255660 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:01.255687 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:01.257875 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:01.260999 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:01.261005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:01.261019 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:01.261434 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:01.261818 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:01.261821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:01.261823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:01.262006 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:01.262504 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:01.262509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:01.262831 (+ 322us) client_negotiation.cc:769] Sending connection context
0926 19:38:01.263057 (+ 226us) client_negotiation.cc:240] Negotiation successful
0926 19:38:01.263428 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":335,"threads_started":1}
I20260926 19:38:01.266103 25694 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42769
I20260926 19:38:01.266433 25694 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:01.267261 25694 heartbeater.cc:511] Master 127.6.215.254:42769 requested a full tablet report, sending...
I20260926 19:38:01.268694 25237 ts_manager.cc:194] Registered new tserver with Master: d32d6e965a5f43cda1f8ee7d37ee584f (127.6.215.195:40993)
I20260926 19:38:01.269516 25237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:54521
I20260926 19:38:01.279672 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:38:01.296530 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.283946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51102 (local address 127.6.215.254:42769)
0926 19:38:01.284271 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.284278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.284477 (+ 199us) server_negotiation.cc:407] Connection header received
0926 19:38:01.284629 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.284633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.284697 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.284827 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:01.286236 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.286944 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.288098 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.288535 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.288641 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.291254 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:01.291259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:01.291263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:01.291292 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:01.293904 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.294645 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.294649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.294651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.294756 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.295145 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.295147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.295149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.295456 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:01.295665 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.296025 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.296159 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260926 19:38:01.300439 25237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:38:01.307384 25237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:38:01.330464 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.319722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48824 (local address 127.6.215.194:37307)
0926 19:38:01.321704 (+ 1982us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.321714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.321746 (+ 32us) server_negotiation.cc:407] Connection header received
0926 19:38:01.321853 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.321861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.322134 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.322339 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:01.323172 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.324899 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.327734 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.329144 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.329291 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.329498 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.329934 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.330122 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1807,"thread_start_us":1698,"threads_started":1}
I20260926 19:38:01.330482 25708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.319633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37307 (local address 127.0.0.1:48824)
0926 19:38:01.320403 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:01.320441 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:01.320631 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:01.322640 (+ 2009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:01.322650 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:01.322681 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:01.323003 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.323013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.325024 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.325030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.327524 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.327541 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.329363 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.329373 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.329664 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.329725 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 19:38:01.329868 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 19:38:01.330057 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":45,"thread_start_us":2111,"threads_started":1}
I20260926 19:38:01.331606 25707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.319673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:40993 (local address 127.0.0.1:36498)
0926 19:38:01.321996 (+ 2323us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:01.322016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:01.322138 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:01.324114 (+ 1976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:01.324122 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:01.324158 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:01.324680 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.324699 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.326501 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.326507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.328704 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.328720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.330818 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.330824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.331003 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.331068 (+ 65us) client_negotiation.cc:769] Sending connection context
0926 19:38:01.331151 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 19:38:01.331283 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2214,"thread_start_us":688,"threads_started":1}
I20260926 19:38:01.332438 25711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.321208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:32821 (local address 127.0.0.1:36230)
0926 19:38:01.324661 (+ 3453us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:01.324678 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:01.324828 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:01.325437 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:01.325447 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:01.325472 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:01.325949 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.325972 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.327751 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.327758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.329539 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.329555 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.331709 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.331715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.331881 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.331917 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 19:38:01.331997 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 19:38:01.332123 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3320,"mutex_wait_us":26,"thread_start_us":3132,"threads_started":1}
I20260926 19:38:01.332443 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.320222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.6.215.195:40993)
0926 19:38:01.323073 (+ 2851us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.323084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.323124 (+ 40us) server_negotiation.cc:407] Connection header received
0926 19:38:01.323243 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.323253 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.323526 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.323767 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:01.324953 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.326362 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.328920 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.330283 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.330410 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.330672 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.331895 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.332067 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2666,"thread_start_us":2521,"threads_started":1}
I20260926 19:38:01.333184 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.321474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.6.215.193:32821)
0926 19:38:01.322685 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.322693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.324805 (+ 2112us) server_negotiation.cc:407] Connection header received
0926 19:38:01.324969 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.324976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.325239 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.325440 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:01.326151 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.327582 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.329703 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.331188 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.331320 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.331552 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.332644 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.332886 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":893,"threads_started":1}
I20260926 19:38:01.335023 25495 tablet_service.cc:1467] Processing CreateTablet for tablet 0016a8616b9f4803b28b0e304b8ecdf7 (DEFAULT_TABLE table=test-table [id=f9dd2e7e70bc4d009ef04990afbd0986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:01.336534 25495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0016a8616b9f4803b28b0e304b8ecdf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:01.336014 25361 tablet_service.cc:1467] Processing CreateTablet for tablet 0016a8616b9f4803b28b0e304b8ecdf7 (DEFAULT_TABLE table=test-table [id=f9dd2e7e70bc4d009ef04990afbd0986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:01.336210 25629 tablet_service.cc:1467] Processing CreateTablet for tablet 0016a8616b9f4803b28b0e304b8ecdf7 (DEFAULT_TABLE table=test-table [id=f9dd2e7e70bc4d009ef04990afbd0986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:01.337446 25361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0016a8616b9f4803b28b0e304b8ecdf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:01.337538 25629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0016a8616b9f4803b28b0e304b8ecdf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:01.349226 25713 tablet_bootstrap.cc:492] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Bootstrap starting.
I20260926 19:38:01.353515 25714 tablet_bootstrap.cc:492] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Bootstrap starting.
I20260926 19:38:01.353817 25713 tablet_bootstrap.cc:654] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:01.355216 25713 log.cc:824] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:01.356734 25715 tablet_bootstrap.cc:492] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Bootstrap starting.
I20260926 19:38:01.357136 25714 tablet_bootstrap.cc:654] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:01.358399 25714 log.cc:824] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:01.358708 25713 tablet_bootstrap.cc:492] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: No bootstrap required, opened a new log
I20260926 19:38:01.359004 25713 ts_tablet_manager.cc:1397] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 19:38:01.360538 25715 tablet_bootstrap.cc:654] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:01.362082 25715 log.cc:824] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:01.364565 25714 tablet_bootstrap.cc:492] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: No bootstrap required, opened a new log
I20260926 19:38:01.364915 25714 ts_tablet_manager.cc:1397] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260926 19:38:01.365167 25715 tablet_bootstrap.cc:492] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: No bootstrap required, opened a new log
I20260926 19:38:01.365425 25715 ts_tablet_manager.cc:1397] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 19:38:01.368606 25713 raft_consensus.cc:348] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.369122 25713 raft_consensus.cc:374] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:01.369285 25713 raft_consensus.cc:729] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e78d35f76ba4868aca5c55ff3d548c0, State: Initialized, Role: FOLLOWER
I20260926 19:38:01.370010 25713 consensus_queue.cc:260] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.372424 25713 ts_tablet_manager.cc:1428] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260926 19:38:01.373236 25714 raft_consensus.cc:348] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.373697 25714 raft_consensus.cc:374] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:01.373857 25714 raft_consensus.cc:729] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c2254ea726440aa1801fb249b14d5f, State: Initialized, Role: FOLLOWER
I20260926 19:38:01.374459 25714 consensus_queue.cc:260] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.375756 25560 heartbeater.cc:503] Master 127.6.215.254:42769 was elected leader, sending a full tablet report...
I20260926 19:38:01.376338 25714 ts_tablet_manager.cc:1428] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260926 19:38:01.376606 25715 raft_consensus.cc:348] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.377132 25715 raft_consensus.cc:374] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:01.377270 25715 raft_consensus.cc:729] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d32d6e965a5f43cda1f8ee7d37ee584f, State: Initialized, Role: FOLLOWER
I20260926 19:38:01.378026 25715 consensus_queue.cc:260] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.379333 25694 heartbeater.cc:503] Master 127.6.215.254:42769 was elected leader, sending a full tablet report...
I20260926 19:38:01.380024 25715 ts_tablet_manager.cc:1428] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260926 19:38:01.449339 25427 tablet.cc:2369] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:01.450289 25721 raft_consensus.cc:482] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:38:01.450532 25721 raft_consensus.cc:504] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.452046 25721 leader_election.cc:288] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03c2254ea726440aa1801fb249b14d5f (127.6.215.194:37307), 9e78d35f76ba4868aca5c55ff3d548c0 (127.6.215.193:32821)
I20260926 19:38:01.457978 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.452626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:38411 (local address 127.6.215.193:32821)
0926 19:38:01.452862 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.452870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.453023 (+ 153us) server_negotiation.cc:407] Connection header received
0926 19:38:01.453247 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.453257 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.453317 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.453415 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:01.453832 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.454548 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.456179 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.457088 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.457260 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.457340 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.457598 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.457724 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:38:01.458082 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.452549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:32821 (local address 127.6.215.195:38411)
0926 19:38:01.452918 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:01.452934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:01.453096 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:01.453436 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:01.453440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:01.453467 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:01.453692 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.453701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.454684 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.454691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.456011 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.456022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.457249 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.457255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.457440 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.457463 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 19:38:01.457571 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 19:38:01.457714 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":1,"spinlock_wait_cycles":23424}
I20260926 19:38:01.459091 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.452619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:37307 (local address 127.6.215.195:54395)
0926 19:38:01.453385 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:01.453399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:01.453575 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:01.453904 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:01.453909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:01.453932 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:01.454230 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.454238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.455234 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.455238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.456869 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:01.456880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.457985 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.457995 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:01.458503 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.458534 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:38:01.458645 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 19:38:01.458757 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"spinlock_wait_cycles":896,"thread_start_us":490,"threads_started":1}
I20260926 19:38:01.459124 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.452613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54395 (local address 127.6.215.194:37307)
0926 19:38:01.452839 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.452845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.453495 (+ 650us) server_negotiation.cc:407] Connection header received
0926 19:38:01.453734 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.453738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.453795 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.453890 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:01.454387 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.455068 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.457014 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.457789 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.457973 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.458041 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.458712 (+ 671us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.458855 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 19:38:01.459197 25381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0016a8616b9f4803b28b0e304b8ecdf7" candidate_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" is_pre_election: true
I20260926 19:38:01.459790 25381 raft_consensus.cc:2454] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d32d6e965a5f43cda1f8ee7d37ee584f in term 0.
I20260926 19:38:01.460191 25515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0016a8616b9f4803b28b0e304b8ecdf7" candidate_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c2254ea726440aa1801fb249b14d5f" is_pre_election: true
I20260926 19:38:01.460604 25583 leader_election.cc:302] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e78d35f76ba4868aca5c55ff3d548c0, d32d6e965a5f43cda1f8ee7d37ee584f; no voters:
I20260926 19:38:01.460726 25515 raft_consensus.cc:2454] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d32d6e965a5f43cda1f8ee7d37ee584f in term 0.
I20260926 19:38:01.461069 25721 raft_consensus.cc:2781] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:38:01.461253 25721 raft_consensus.cc:482] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:38:01.461393 25721 raft_consensus.cc:3037] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:01.463606 25721 raft_consensus.cc:504] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.464210 25721 leader_election.cc:288] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [CANDIDATE]: Term 1 election: Requested vote from peers 03c2254ea726440aa1801fb249b14d5f (127.6.215.194:37307), 9e78d35f76ba4868aca5c55ff3d548c0 (127.6.215.193:32821)
I20260926 19:38:01.464715 25515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0016a8616b9f4803b28b0e304b8ecdf7" candidate_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c2254ea726440aa1801fb249b14d5f"
I20260926 19:38:01.464715 25381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0016a8616b9f4803b28b0e304b8ecdf7" candidate_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e78d35f76ba4868aca5c55ff3d548c0"
I20260926 19:38:01.464954 25515 raft_consensus.cc:3037] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:01.464954 25381 raft_consensus.cc:3037] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:01.468187 25515 raft_consensus.cc:2454] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d32d6e965a5f43cda1f8ee7d37ee584f in term 1.
I20260926 19:38:01.468187 25381 raft_consensus.cc:2454] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d32d6e965a5f43cda1f8ee7d37ee584f in term 1.
I20260926 19:38:01.468657 25583 leader_election.cc:302] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e78d35f76ba4868aca5c55ff3d548c0, d32d6e965a5f43cda1f8ee7d37ee584f; no voters:
I20260926 19:38:01.468976 25721 raft_consensus.cc:2781] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:01.469435 25721 raft_consensus.cc:686] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 1 LEADER]: Becoming Leader. State: Replica: d32d6e965a5f43cda1f8ee7d37ee584f, State: Running, Role: LEADER
I20260926 19:38:01.469911 25721 consensus_queue.cc:237] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } }
I20260926 19:38:01.476086 25236 catalog_manager.cc:5495] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f reported cstate change: term changed from 0 to 1, leader changed from <none> to d32d6e965a5f43cda1f8ee7d37ee584f (127.6.215.195). New cstate: current_term: 1 leader_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d32d6e965a5f43cda1f8ee7d37ee584f" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 40993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 } health_report { overall_health: UNKNOWN } } }
W20260926 19:38:01.477123 25561 tablet.cc:2369] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:38:01.494681 25695 tablet.cc:2369] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:01.523559 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.517983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36512 (local address 127.6.215.195:40993)
0926 19:38:01.518186 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.518191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.518210 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:38:01.518427 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.518431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.518516 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.518674 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:01.519054 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.519781 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.521263 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.521721 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.521858 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.522478 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:01.522951 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:01.523137 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.523294 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.523340 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:38:01.532648 25381 raft_consensus.cc:1261] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 1 FOLLOWER]: Refusing update from remote peer d32d6e965a5f43cda1f8ee7d37ee584f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:01.532648 25515 raft_consensus.cc:1261] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 1 FOLLOWER]: Refusing update from remote peer d32d6e965a5f43cda1f8ee7d37ee584f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:01.533761 25721 consensus_queue.cc:1035] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [LEADER]: Connected to new peer: Peer: permanent_uuid: "03c2254ea726440aa1801fb249b14d5f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:01.534060 25723 consensus_queue.cc:1035] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e78d35f76ba4868aca5c55ff3d548c0" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 32821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:01.548460 25727 mvcc.cc:205] Tried to move back new op lower bound from 7333689268345077760 to 7333689268104323072. Current Snapshot: MvccSnapshot[applied={T|T < 7333689268345077760}]
I20260926 19:38:01.548600 25728 mvcc.cc:205] Tried to move back new op lower bound from 7333689268345077760 to 7333689268104323072. Current Snapshot: MvccSnapshot[applied={T|T < 7333689268345077760}]
I20260926 19:38:01.557150 25236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:01.557451 25236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:01.561092 25236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69962ee3c6624772958b402d224462e7: Sending DeleteTablet for 3 replicas of tablet 0016a8616b9f4803b28b0e304b8ecdf7
I20260926 19:38:01.562364 25629 tablet_service.cc:1514] Processing DeleteTablet for tablet 0016a8616b9f4803b28b0e304b8ecdf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:01 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:36498
I20260926 19:38:01.562356 25495 tablet_service.cc:1514] Processing DeleteTablet for tablet 0016a8616b9f4803b28b0e304b8ecdf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:01 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:48824
I20260926 19:38:01.562595 25361 tablet_service.cc:1514] Processing DeleteTablet for tablet 0016a8616b9f4803b28b0e304b8ecdf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:01 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:36230
I20260926 19:38:01.566848 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25298
I20260926 19:38:01.581605 25736 tablet_replica.cc:330] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: stopping tablet replica
I20260926 19:38:01.581765 25737 tablet_replica.cc:330] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: stopping tablet replica
I20260926 19:38:01.582264 25736 raft_consensus.cc:2229] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:38:01.582259 25737 raft_consensus.cc:2229] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:01.582818 25737 raft_consensus.cc:2258] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:01.585291 25736 raft_consensus.cc:2258] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:01.587510 25737 ts_tablet_manager.cc:1905] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:01.588374 25735 tablet_replica.cc:330] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: stopping tablet replica
I20260926 19:38:01.590225 25736 ts_tablet_manager.cc:1905] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:01.590781 25735 raft_consensus.cc:2229] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:01.594844 25737 ts_tablet_manager.cc:1918] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:01.595031 25737 log.cc:1196] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/0016a8616b9f4803b28b0e304b8ecdf7
I20260926 19:38:01.595593 25737 ts_tablet_manager.cc:1939] T 0016a8616b9f4803b28b0e304b8ecdf7 P 9e78d35f76ba4868aca5c55ff3d548c0: Deleting consensus metadata
I20260926 19:38:01.596120 25735 raft_consensus.cc:2258] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 26 19:38:01 dist-test-slave-phbh krb5kdc[25192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:38:01.597568 25222 catalog_manager.cc:4841] TS 9e78d35f76ba4868aca5c55ff3d548c0 (127.6.215.193:32821): tablet 0016a8616b9f4803b28b0e304b8ecdf7 (table test-table [id=f9dd2e7e70bc4d009ef04990afbd0986]) successfully deleted
I20260926 19:38:01.601146 25735 ts_tablet_manager.cc:1905] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:01.602166 25736 ts_tablet_manager.cc:1918] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:01.602344 25736 log.cc:1196] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/0016a8616b9f4803b28b0e304b8ecdf7
I20260926 19:38:01.602927 25736 ts_tablet_manager.cc:1939] T 0016a8616b9f4803b28b0e304b8ecdf7 P d32d6e965a5f43cda1f8ee7d37ee584f: Deleting consensus metadata
I20260926 19:38:01.604677 25222 catalog_manager.cc:4841] TS d32d6e965a5f43cda1f8ee7d37ee584f (127.6.215.195:40993): tablet 0016a8616b9f4803b28b0e304b8ecdf7 (table test-table [id=f9dd2e7e70bc4d009ef04990afbd0986]) successfully deleted
I20260926 19:38:01.607533 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.568558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.6.215.193:32821)
0926 19:38:01.568807 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.568815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.571149 (+ 2334us) server_negotiation.cc:407] Connection header received
0926 19:38:01.571233 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.571243 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.571383 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.571497 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:01.575561 (+ 4064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.577141 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.582563 (+ 5422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.583286 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.583424 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.599430 (+ 16006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:01.599444 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:01.599462 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:01.599511 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:01.603379 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.605018 (+ 1639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.605027 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.605031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.605191 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.605633 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.605638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.605642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.606107 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:01.606379 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.606930 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.607177 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 19:38:01.608042 25735 ts_tablet_manager.cc:1918] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:01.608213 25735 log.cc:1196] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/0016a8616b9f4803b28b0e304b8ecdf7
I20260926 19:38:01.608762 25735 ts_tablet_manager.cc:1939] T 0016a8616b9f4803b28b0e304b8ecdf7 P 03c2254ea726440aa1801fb249b14d5f: Deleting consensus metadata
I20260926 19:38:01.609002 25421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:01.610697 25224 catalog_manager.cc:4841] TS 03c2254ea726440aa1801fb249b14d5f (127.6.215.194:37307): tablet 0016a8616b9f4803b28b0e304b8ecdf7 (table test-table [id=f9dd2e7e70bc4d009ef04990afbd0986]) successfully deleted
I20260926 19:38:01.717954 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25298
I20260926 19:38:01.736497 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25432
Sep 26 19:38:01 dist-test-slave-phbh krb5kdc[25192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:38:01.752914 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.737333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.6.215.194:37307)
0926 19:38:01.737844 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.737848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.737866 (+ 18us) server_negotiation.cc:407] Connection header received
0926 19:38:01.737915 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.737919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.737977 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.738094 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:01.739402 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.740568 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.742106 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.742496 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.742633 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.746933 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:01.746951 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:01.746964 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:01.746997 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:01.750034 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.750723 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.750732 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.750736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.750863 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.751255 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.751261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.751265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.751655 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:01.751952 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.752286 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.752506 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406}
I20260926 19:38:01.753729 25555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:01.843765 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25432
I20260926 19:38:01.861666 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25566
Sep 26 19:38:01 dist-test-slave-phbh krb5kdc[25192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:38:01.879801 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:01.862478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.6.215.195:40993)
0926 19:38:01.862692 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:01.862701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:01.862721 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:38:01.862823 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:01.862828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:01.862906 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:01.863026 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:01.864308 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.865260 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.866531 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:01.867016 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:01.867160 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:01.871849 (+ 4689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:01.871869 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:01.871892 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:01.871953 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:01.876102 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.877325 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.877340 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.877345 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.877571 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:01.877998 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:01.878003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:01.878007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:01.878485 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:01.878850 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:01.879108 (+ 258us) server_negotiation.cc:299] Negotiation successful
0926 19:38:01.879355 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:38:01.880722 25689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:02.003109 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25566
I20260926 19:38:02.025409 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25205
I20260926 19:38:02.026528 25267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:02.138724 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25205
2026-09-26T19:38:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4389 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T19:38:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:02Z Disabled control of system clock
I20260926 19:38:02.214807 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40647
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:40647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 19:38:02.544222 25746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:02.544531 25746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:02.544669 25746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:02.551999 25746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:02.552114 25746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:02.552191 25746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:02.552302 25746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:02.552383 25746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:02.561843 25746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:40647
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:02.563162 25746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:02.564953 25746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:02.576673 25754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:02.576709 25752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:02.576845 25751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:02.577659 25746 server_base.cc:1034] running on GCE node
I20260926 19:38:02.578393 25746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:02.579715 25746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:02.580912 25746 hybrid_clock.cc:648] HybridClock initialized: now 1790451482580870 us; error 42 us; skew 500 ppm
I20260926 19:38:02.581321 25746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:02.583542 25746 webserver.cc:460] Webserver started at http://127.6.215.254:39831/ using document root <none> and password file <none>
I20260926 19:38:02.584168 25746 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:02.584251 25746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:02.584498 25746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:02.587440 25746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "49af3319bb0f47fe9ac79f6c6b440ef0"
format_stamp: "Formatted at 2026-09-26 19:38:02 on dist-test-slave-phbh"
server_key: "91c9f24689224b1cff4bba597dd0ea82"
server_key_iv: "b9c29bea3468070981abb2ce4cf42f90"
server_key_version: "encryptionkey@0"
I20260926 19:38:02.588250 25746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "49af3319bb0f47fe9ac79f6c6b440ef0"
format_stamp: "Formatted at 2026-09-26 19:38:02 on dist-test-slave-phbh"
server_key: "91c9f24689224b1cff4bba597dd0ea82"
server_key_iv: "b9c29bea3468070981abb2ce4cf42f90"
server_key_version: "encryptionkey@0"
I20260926 19:38:02.593114 25746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 19:38:02.596469 25760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:02.597983 25746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:02.598188 25746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "49af3319bb0f47fe9ac79f6c6b440ef0"
format_stamp: "Formatted at 2026-09-26 19:38:02 on dist-test-slave-phbh"
server_key: "91c9f24689224b1cff4bba597dd0ea82"
server_key_iv: "b9c29bea3468070981abb2ce4cf42f90"
server_key_version: "encryptionkey@0"
I20260926 19:38:02.598416 25746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:02.611289 25746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:02.612123 25746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:02.612423 25746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:02.631814 25746 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:40647
I20260926 19:38:02.631821 25811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:40647 every 8 connection(s)
I20260926 19:38:02.633241 25746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:02.637012 25812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:02.638095 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25746
I20260926 19:38:02.638310 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:02.638821 7007 external_mini_cluster.cc:1428] Setting key bbe3d86ca3086136d561907357fac0a8
I20260926 19:38:02.649843 25812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0: Bootstrap starting.
I20260926 19:38:02.652943 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:02.640828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.6.215.254:40647)
0926 19:38:02.641903 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:02.641915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:02.641959 (+ 44us) server_negotiation.cc:407] Connection header received
0926 19:38:02.643778 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:02.643801 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:02.644320 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:02.644751 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:02.645385 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:02.646762 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:02.648286 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:02.649100 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:02.649322 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:02.650586 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:02.650594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:02.650607 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:02.651190 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:02.651284 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:02.651305 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:02.651434 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:02.651695 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 19:38:02.651989 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":371,"threads_started":1}
I20260926 19:38:02.653235 25812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:02.654352 25812 log.cc:824] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:02.657173 25812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0: No bootstrap required, opened a new log
I20260926 19:38:02.664373 25812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40647 } }
I20260926 19:38:02.664737 25812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:02.664875 25812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49af3319bb0f47fe9ac79f6c6b440ef0, State: Initialized, Role: FOLLOWER
I20260926 19:38:02.665443 25812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40647 } }
I20260926 19:38:02.665705 25812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:02.665868 25812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:02.666042 25812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:02.668200 25812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40647 } }
I20260926 19:38:02.668691 25812 leader_election.cc:302] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49af3319bb0f47fe9ac79f6c6b440ef0; no voters:
I20260926 19:38:02.669379 25812 leader_election.cc:288] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:02.669579 25817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:02.670277 25817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [term 1 LEADER]: Becoming Leader. State: Replica: 49af3319bb0f47fe9ac79f6c6b440ef0, State: Running, Role: LEADER
I20260926 19:38:02.670861 25817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40647 } }
I20260926 19:38:02.671657 25812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:02.673841 25818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40647 } } }
I20260926 19:38:02.673813 25819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49af3319bb0f47fe9ac79f6c6b440ef0. Latest consensus state: current_term: 1 leader_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49af3319bb0f47fe9ac79f6c6b440ef0" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40647 } } }
I20260926 19:38:02.674331 25818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:02.674407 25819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:02.676326 25824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:02.681643 25824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:02.690348 25824 catalog_manager.cc:1347] Generated new cluster ID: d5b06cdedbbe42eaadc16db06ceadb51
I20260926 19:38:02.690474 25824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:02.707958 25824 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:02.709889 25824 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:02.719872 25824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0: Generated new TSK 0
I20260926 19:38:02.720973 25824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:02.725494 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40647
--builtin_ntp_servers=127.6.215.212:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 19:38:03.057989 25836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:03.058286 25836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:03.058452 25836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:03.066075 25836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:03.066186 25836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:03.066377 25836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:38:03.076011 25836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:03.077464 25836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:03.079303 25836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:03.092710 25841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:03.092804 25842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:03.093061 25844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:03.093847 25836 server_base.cc:1034] running on GCE node
I20260926 19:38:03.094429 25836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:03.095350 25836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:03.096554 25836 hybrid_clock.cc:648] HybridClock initialized: now 1790451483096496 us; error 48 us; skew 500 ppm
I20260926 19:38:03.096931 25836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:03.099211 25836 webserver.cc:460] Webserver started at http://127.6.215.193:40975/ using document root <none> and password file <none>
I20260926 19:38:03.099920 25836 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:03.100039 25836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:03.100380 25836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:03.103549 25836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "fb569a5082dc4a35b6a40aa8eb4b091e"
format_stamp: "Formatted at 2026-09-26 19:38:03 on dist-test-slave-phbh"
server_key: "b4ea667993995a6068f9388ca0a4be2a"
server_key_iv: "60a9abc085ccae0c9b0e97e095f783b6"
server_key_version: "encryptionkey@0"
I20260926 19:38:03.104352 25836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "fb569a5082dc4a35b6a40aa8eb4b091e"
format_stamp: "Formatted at 2026-09-26 19:38:03 on dist-test-slave-phbh"
server_key: "b4ea667993995a6068f9388ca0a4be2a"
server_key_iv: "60a9abc085ccae0c9b0e97e095f783b6"
server_key_version: "encryptionkey@0"
I20260926 19:38:03.109594 25836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260926 19:38:03.113137 25850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:03.114434 25836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:38:03.114607 25836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "fb569a5082dc4a35b6a40aa8eb4b091e"
format_stamp: "Formatted at 2026-09-26 19:38:03 on dist-test-slave-phbh"
server_key: "b4ea667993995a6068f9388ca0a4be2a"
server_key_iv: "60a9abc085ccae0c9b0e97e095f783b6"
server_key_version: "encryptionkey@0"
I20260926 19:38:03.114831 25836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:03.133828 25836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:03.134717 25836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:03.135043 25836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:03.136097 25836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:03.137734 25836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:03.137830 25836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:03.137979 25836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:03.138079 25836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:03.171432 25836 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:39717
I20260926 19:38:03.171489 25962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:39717 every 8 connection(s)
I20260926 19:38:03.172953 25836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:38:03.179956 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25836
I20260926 19:38:03.180145 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:38:03.180577 7007 external_mini_cluster.cc:1428] Setting key 9ec04c53b9b3704a42d312a68a8e9400
I20260926 19:38:03.183553 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40647
--builtin_ntp_servers=127.6.215.212:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 19:38:03.186486 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:03.175454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:40887 (local address 127.6.215.254:40647)
0926 19:38:03.175908 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:03.175913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:03.177148 (+ 1235us) server_negotiation.cc:407] Connection header received
0926 19:38:03.178939 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:03.178945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:03.179037 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:03.179169 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:03.180429 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.181276 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.183315 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.184016 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.184165 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:03.185532 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:03.185542 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:03.185550 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:03.185576 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:03.185643 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:03.185662 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:03.185758 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:03.185960 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 19:38:03.186069 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":251,"threads_started":1}
I20260926 19:38:03.187312 25966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:03.175941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40647 (local address 127.6.215.193:40887)
0926 19:38:03.176946 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:03.176989 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:03.178661 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:03.179440 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:03.179448 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:03.179555 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:03.180259 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:03.180280 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.181465 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.181469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:03.183164 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:03.183177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.184263 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.184270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:03.184472 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:03.185203 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:38:03.185205 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:03.185247 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:38:03.185331 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:03.185762 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:03.185769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:03.185775 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 19:38:03.185916 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 19:38:03.186225 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":296,"threads_started":1}
I20260926 19:38:03.188830 25963 heartbeater.cc:347] Connected to a master server at 127.6.215.254:40647
I20260926 19:38:03.189150 25963 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:03.190099 25963 heartbeater.cc:511] Master 127.6.215.254:40647 requested a full tablet report, sending...
I20260926 19:38:03.192291 25777 ts_manager.cc:194] Registered new tserver with Master: fb569a5082dc4a35b6a40aa8eb4b091e (127.6.215.193:39717)
I20260926 19:38:03.193840 25777 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.193:40887
W20260926 19:38:03.520735 25967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:03.521322 25967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:03.521497 25967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:03.528921 25967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:03.529042 25967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:03.529160 25967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:38:03.538291 25967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:03.539599 25967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:03.541378 25967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:03.554450 25972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:03.554540 25975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:03.554469 25973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:03.555446 25967 server_base.cc:1034] running on GCE node
I20260926 19:38:03.556082 25967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:03.557099 25967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:03.558342 25967 hybrid_clock.cc:648] HybridClock initialized: now 1790451483558275 us; error 53 us; skew 500 ppm
I20260926 19:38:03.558676 25967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:03.561189 25967 webserver.cc:460] Webserver started at http://127.6.215.194:39921/ using document root <none> and password file <none>
I20260926 19:38:03.561879 25967 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:03.562013 25967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:03.562342 25967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:03.565223 25967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "d96e9408541a4e03aae7bc2b5f407a66"
format_stamp: "Formatted at 2026-09-26 19:38:03 on dist-test-slave-phbh"
server_key: "3f965d85a24e0ee1605c2729a93c804b"
server_key_iv: "aa1e41b4dc7181ccddfb06fff9d4cd1b"
server_key_version: "encryptionkey@0"
I20260926 19:38:03.565997 25967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "d96e9408541a4e03aae7bc2b5f407a66"
format_stamp: "Formatted at 2026-09-26 19:38:03 on dist-test-slave-phbh"
server_key: "3f965d85a24e0ee1605c2729a93c804b"
server_key_iv: "aa1e41b4dc7181ccddfb06fff9d4cd1b"
server_key_version: "encryptionkey@0"
I20260926 19:38:03.571002 25967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:03.574481 25981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:03.575778 25967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:38:03.575963 25967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "d96e9408541a4e03aae7bc2b5f407a66"
format_stamp: "Formatted at 2026-09-26 19:38:03 on dist-test-slave-phbh"
server_key: "3f965d85a24e0ee1605c2729a93c804b"
server_key_iv: "aa1e41b4dc7181ccddfb06fff9d4cd1b"
server_key_version: "encryptionkey@0"
I20260926 19:38:03.576191 25967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:03.618826 25967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:03.619791 25967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:03.620091 25967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:03.621137 25967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:03.622558 25967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:03.622687 25967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:03.622820 25967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:03.622913 25967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:03.658556 25967 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:44761
I20260926 19:38:03.658603 26093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:44761 every 8 connection(s)
I20260926 19:38:03.659826 25967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:38:03.661450 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 25967
I20260926 19:38:03.661670 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:38:03.662209 7007 external_mini_cluster.cc:1428] Setting key 15bc77af886424cb4a760d038316aa61
I20260926 19:38:03.665241 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40647
--builtin_ntp_servers=127.6.215.212:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 19:38:03.672493 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:03.662504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:41859 (local address 127.6.215.254:40647)
0926 19:38:03.662720 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:03.662729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:03.664284 (+ 1555us) server_negotiation.cc:407] Connection header received
0926 19:38:03.666213 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:03.666218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:03.666303 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:03.666411 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:03.667420 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.668134 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.669754 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.670147 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.670283 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:03.671606 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:03.671611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:03.671615 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:03.671631 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:03.671682 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:03.671696 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:03.671794 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:03.672083 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 19:38:03.672189 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 19:38:03.673575 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:03.663135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40647 (local address 127.6.215.194:41859)
0926 19:38:03.664118 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:03.664178 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:03.665956 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:03.666639 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:03.666647 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:03.666730 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:03.667255 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:03.667271 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.668291 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.668298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:03.669599 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:03.669612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:03.670366 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:03.670372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:03.670602 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:03.671294 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:38:03.671296 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:03.671338 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:38:03.671408 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:03.671791 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:03.671803 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:03.671812 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 19:38:03.672081 (+ 269us) client_negotiation.cc:240] Negotiation successful
0926 19:38:03.672448 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":318,"threads_started":1}
I20260926 19:38:03.674978 26094 heartbeater.cc:347] Connected to a master server at 127.6.215.254:40647
I20260926 19:38:03.675278 26094 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:03.676168 26094 heartbeater.cc:511] Master 127.6.215.254:40647 requested a full tablet report, sending...
I20260926 19:38:03.677812 25777 ts_manager.cc:194] Registered new tserver with Master: d96e9408541a4e03aae7bc2b5f407a66 (127.6.215.194:44761)
I20260926 19:38:03.678620 25777 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.194:41859
W20260926 19:38:03.990952 26097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:03.991320 26097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:03.991488 26097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:03.998493 26097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:03.998580 26097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:03.998754 26097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:38:04.007884 26097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:40647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:04.009163 26097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:04.010900 26097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:04.024747 26102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:04.024793 26105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:04.025046 26103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:04.025806 26097 server_base.cc:1034] running on GCE node
I20260926 19:38:04.026515 26097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:04.027480 26097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:04.028738 26097 hybrid_clock.cc:648] HybridClock initialized: now 1790451484028687 us; error 40 us; skew 500 ppm
I20260926 19:38:04.029153 26097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:04.031713 26097 webserver.cc:460] Webserver started at http://127.6.215.195:44507/ using document root <none> and password file <none>
I20260926 19:38:04.032459 26097 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:04.032606 26097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:04.033026 26097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:04.036017 26097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "761f7a98d94c43fab8913d220b439578"
format_stamp: "Formatted at 2026-09-26 19:38:04 on dist-test-slave-phbh"
server_key: "a3721cfb7f395e70d9326862a5f96539"
server_key_iv: "d999ced8e42d8ab492a6f84d61fe6e90"
server_key_version: "encryptionkey@0"
I20260926 19:38:04.036921 26097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "761f7a98d94c43fab8913d220b439578"
format_stamp: "Formatted at 2026-09-26 19:38:04 on dist-test-slave-phbh"
server_key: "a3721cfb7f395e70d9326862a5f96539"
server_key_iv: "d999ced8e42d8ab492a6f84d61fe6e90"
server_key_version: "encryptionkey@0"
I20260926 19:38:04.042179 26097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:38:04.045682 26111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:04.047145 26097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:04.047349 26097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "761f7a98d94c43fab8913d220b439578"
format_stamp: "Formatted at 2026-09-26 19:38:04 on dist-test-slave-phbh"
server_key: "a3721cfb7f395e70d9326862a5f96539"
server_key_iv: "d999ced8e42d8ab492a6f84d61fe6e90"
server_key_version: "encryptionkey@0"
I20260926 19:38:04.047605 26097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:04.065742 26097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:04.066702 26097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:04.067061 26097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:04.068118 26097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:04.069779 26097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:04.069938 26097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:04.070092 26097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:04.070191 26097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:04.106179 26097 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:44223
I20260926 19:38:04.106226 26223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:44223 every 8 connection(s)
I20260926 19:38:04.107601 26097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:38:04.111508 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 26097
I20260926 19:38:04.111694 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:38:04.112170 7007 external_mini_cluster.cc:1428] Setting key 895836d15513745af31842488fd34f13
I20260926 19:38:04.120575 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.110252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:34623 (local address 127.6.215.254:40647)
0926 19:38:04.110457 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.110462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.111937 (+ 1475us) server_negotiation.cc:407] Connection header received
0926 19:38:04.114174 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.114179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.114271 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.114388 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:04.115552 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.116302 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.117971 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.118406 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.118553 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.119930 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:04.119935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:04.119938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:04.119955 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:04.119984 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:04.119996 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:04.120065 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.120276 (+ 211us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.120350 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 19:38:04.121632 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.110734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40647 (local address 127.6.215.195:34623)
0926 19:38:04.111755 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.111809 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:04.113882 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:04.114624 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:04.114633 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:04.114718 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:04.115389 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.115408 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.116444 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.116456 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.117823 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.117835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.118678 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.118683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.118903 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.119581 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:38:04.119584 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:04.119666 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:38:04.119760 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:04.120085 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:04.120093 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:04.120099 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 19:38:04.120249 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 19:38:04.120523 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":318,"threads_started":1}
I20260926 19:38:04.122850 26224 heartbeater.cc:347] Connected to a master server at 127.6.215.254:40647
I20260926 19:38:04.123091 26224 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:04.123749 26224 heartbeater.cc:511] Master 127.6.215.254:40647 requested a full tablet report, sending...
I20260926 19:38:04.125048 25777 ts_manager.cc:194] Registered new tserver with Master: 761f7a98d94c43fab8913d220b439578 (127.6.215.195:44223)
I20260926 19:38:04.126005 25777 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.195:34623
I20260926 19:38:04.127584 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:38:04.137825 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.131950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.6.215.254:40647)
0926 19:38:04.132233 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.132238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.132443 (+ 205us) server_negotiation.cc:407] Connection header received
0926 19:38:04.132564 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.132568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.132639 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.132720 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:04.133346 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.133998 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.135517 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.135903 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.135995 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.137095 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:04.137100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:04.137103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:04.137120 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:04.137159 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:04.137171 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:04.137281 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.137431 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.137505 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 19:38:04.141579 25777 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:38:04.148239 25777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:38:04.197156 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.160851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.6.215.194:44761)
0926 19:38:04.188050 (+ 27199us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.188062 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.188104 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:38:04.188243 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.188257 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.188594 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.188868 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:04.190334 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.191730 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.194740 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.195964 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.196119 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.196375 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.196632 (+ 257us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.196834 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26990,"thread_start_us":26848,"threads_started":1}
I20260926 19:38:04.197220 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.160487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:44761 (local address 127.0.0.1:49308)
0926 19:38:04.170516 (+ 10029us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.170562 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:04.187207 (+ 16645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:04.189202 (+ 1995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:04.189214 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:04.189260 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:04.189834 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.189850 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.191887 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.191895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.194503 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.194516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.196165 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.196171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.196370 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.196431 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 19:38:04.196579 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 19:38:04.196769 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9730,"mutex_wait_us":46,"thread_start_us":27035,"threads_started":1}
I20260926 19:38:04.197428 25963 heartbeater.cc:503] Master 127.6.215.254:40647 was elected leader, sending a full tablet report...
I20260926 19:38:04.198541 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.160793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52210)
0926 19:38:04.187882 (+ 27089us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.187901 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:04.188061 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:04.189920 (+ 1859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:04.189929 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:04.189967 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:04.190497 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.190509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.192802 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.192808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.195329 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.195343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.197116 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.197124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.197942 (+ 818us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.197982 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 19:38:04.198052 (+ 70us) client_negotiation.cc:240] Negotiation successful
0926 19:38:04.198187 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26932,"mutex_wait_us":195,"thread_start_us":27642,"threads_started":1}
I20260926 19:38:04.199077 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.161383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:44223 (local address 127.0.0.1:35766)
0926 19:38:04.188246 (+ 26863us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.188263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:04.188370 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:04.190335 (+ 1965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:04.190344 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:04.190369 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:04.190830 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.190845 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.192959 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.192964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.195187 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.195202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.197975 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.197981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.198159 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.198186 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 19:38:04.198257 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 19:38:04.198367 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26734,"mutex_wait_us":168,"thread_start_us":8739,"threads_started":1}
I20260926 19:38:04.199903 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.162554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.6.215.195:44223)
0926 19:38:04.182876 (+ 20322us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.182891 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.188858 (+ 5967us) server_negotiation.cc:407] Connection header received
0926 19:38:04.188992 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.188998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.189330 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.189543 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:04.191060 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.192822 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.195376 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.196873 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.197013 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.199331 (+ 2318us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.199401 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.199562 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7943,"thread_start_us":8181,"threads_started":1}
I20260926 19:38:04.200312 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.161908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.6.215.193:39717)
0926 19:38:04.170220 (+ 8312us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.170230 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.188228 (+ 17998us) server_negotiation.cc:407] Connection header received
0926 19:38:04.188342 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.188351 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.188599 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.190706 (+ 2107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:04.190947 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.192628 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.195520 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.196879 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.199099 (+ 2220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.199699 (+ 600us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.199771 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.199933 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8093,"thread_start_us":7950,"threads_started":1}
I20260926 19:38:04.201535 26029 tablet_service.cc:1467] Processing CreateTablet for tablet 2aaff1c67ec64640b420cc66ffce56d0 (DEFAULT_TABLE table=test-table [id=611821e117c144c6968aeddd5c394bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:04.203078 26029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aaff1c67ec64640b420cc66ffce56d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:04.203517 25898 tablet_service.cc:1467] Processing CreateTablet for tablet 2aaff1c67ec64640b420cc66ffce56d0 (DEFAULT_TABLE table=test-table [id=611821e117c144c6968aeddd5c394bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:04.203555 26159 tablet_service.cc:1467] Processing CreateTablet for tablet 2aaff1c67ec64640b420cc66ffce56d0 (DEFAULT_TABLE table=test-table [id=611821e117c144c6968aeddd5c394bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:04.204643 25898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aaff1c67ec64640b420cc66ffce56d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:04.204900 26159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aaff1c67ec64640b420cc66ffce56d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:04.215281 26242 tablet_bootstrap.cc:492] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Bootstrap starting.
I20260926 19:38:04.219518 26242 tablet_bootstrap.cc:654] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:04.220620 26242 log.cc:824] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:04.221170 26244 tablet_bootstrap.cc:492] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Bootstrap starting.
I20260926 19:38:04.222862 26243 tablet_bootstrap.cc:492] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Bootstrap starting.
I20260926 19:38:04.226092 26243 tablet_bootstrap.cc:654] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:04.227092 26243 log.cc:824] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:04.227293 26244 tablet_bootstrap.cc:654] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:04.228657 26244 log.cc:824] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:04.230151 26243 tablet_bootstrap.cc:492] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: No bootstrap required, opened a new log
I20260926 19:38:04.230145 26242 tablet_bootstrap.cc:492] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: No bootstrap required, opened a new log
I20260926 19:38:04.230425 26242 ts_tablet_manager.cc:1397] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260926 19:38:04.230425 26243 ts_tablet_manager.cc:1397] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260926 19:38:04.231428 26244 tablet_bootstrap.cc:492] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: No bootstrap required, opened a new log
I20260926 19:38:04.231698 26244 ts_tablet_manager.cc:1397] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260926 19:38:04.238958 26244 raft_consensus.cc:348] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.239357 26244 raft_consensus.cc:374] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:04.239485 26244 raft_consensus.cc:729] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 761f7a98d94c43fab8913d220b439578, State: Initialized, Role: FOLLOWER
I20260926 19:38:04.240130 26244 consensus_queue.cc:260] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.241322 26224 heartbeater.cc:503] Master 127.6.215.254:40647 was elected leader, sending a full tablet report...
I20260926 19:38:04.241142 26242 raft_consensus.cc:348] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.241142 26243 raft_consensus.cc:348] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.241555 26243 raft_consensus.cc:374] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:04.241555 26242 raft_consensus.cc:374] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:04.241654 26242 raft_consensus.cc:729] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb569a5082dc4a35b6a40aa8eb4b091e, State: Initialized, Role: FOLLOWER
I20260926 19:38:04.241645 26243 raft_consensus.cc:729] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d96e9408541a4e03aae7bc2b5f407a66, State: Initialized, Role: FOLLOWER
I20260926 19:38:04.242211 26244 ts_tablet_manager.cc:1428] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Time spent starting tablet: real 0.010s user 0.009s sys 0.002s
I20260926 19:38:04.242229 26243 consensus_queue.cc:260] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.242242 26242 consensus_queue.cc:260] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.246140 26094 heartbeater.cc:503] Master 127.6.215.254:40647 was elected leader, sending a full tablet report...
I20260926 19:38:04.246832 26243 ts_tablet_manager.cc:1428] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
I20260926 19:38:04.248437 26242 ts_tablet_manager.cc:1428] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Time spent starting tablet: real 0.018s user 0.015s sys 0.003s
W20260926 19:38:04.359491 26225 tablet.cc:2369] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:04.360741 26250 raft_consensus.cc:482] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:38:04.361142 26250 raft_consensus.cc:504] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.362902 26250 leader_election.cc:288] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d96e9408541a4e03aae7bc2b5f407a66 (127.6.215.194:44761), 761f7a98d94c43fab8913d220b439578 (127.6.215.195:44223)
I20260926 19:38:04.369999 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.363370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:44761 (local address 127.6.215.193:53425)
0926 19:38:04.364365 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.364384 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:04.364537 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:04.364896 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:04.364900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:04.364932 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:04.365299 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.365310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.366317 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.366324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.367997 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.368009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.369236 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.369245 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.369434 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.369479 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 19:38:04.369550 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 19:38:04.369650 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"mutex_wait_us":101,"thread_start_us":621,"threads_started":1}
I20260926 19:38:04.370052 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.363370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:53425 (local address 127.6.215.194:44761)
0926 19:38:04.363592 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.363601 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.364500 (+ 899us) server_negotiation.cc:407] Connection header received
0926 19:38:04.364667 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.364670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.364773 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.364880 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:04.365475 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.366181 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.368201 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.369041 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.369172 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.369220 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.369654 (+ 434us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.369779 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 19:38:04.370328 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.363535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:33107 (local address 127.6.215.195:44223)
0926 19:38:04.363790 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.363798 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.364371 (+ 573us) server_negotiation.cc:407] Connection header received
0926 19:38:04.364572 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.364581 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.364663 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.364744 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:04.365545 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.366270 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.368321 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.369196 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.369350 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.369426 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.369941 (+ 515us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.370102 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260926 19:38:04.370471 26251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.363186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:44223 (local address 127.6.215.193:33107)
0926 19:38:04.364234 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.364255 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:04.364403 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:04.364812 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:04.364819 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:04.364858 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:04.365373 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.365383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.366398 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.366401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.368129 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:04.368146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.369375 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.369380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:04.369749 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.369780 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 19:38:04.369895 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 19:38:04.370103 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":234,"thread_start_us":637,"threads_started":1}
I20260926 19:38:04.371414 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aaff1c67ec64640b420cc66ffce56d0" candidate_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "761f7a98d94c43fab8913d220b439578" is_pre_election: true
I20260926 19:38:04.371500 26049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aaff1c67ec64640b420cc66ffce56d0" candidate_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d96e9408541a4e03aae7bc2b5f407a66" is_pre_election: true
I20260926 19:38:04.371948 26179 raft_consensus.cc:2454] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb569a5082dc4a35b6a40aa8eb4b091e in term 0.
I20260926 19:38:04.371999 26049 raft_consensus.cc:2454] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb569a5082dc4a35b6a40aa8eb4b091e in term 0.
I20260926 19:38:04.372607 25853 leader_election.cc:302] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761f7a98d94c43fab8913d220b439578, fb569a5082dc4a35b6a40aa8eb4b091e; no voters:
I20260926 19:38:04.373045 26250 raft_consensus.cc:2781] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:38:04.373191 26250 raft_consensus.cc:482] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:38:04.373286 26250 raft_consensus.cc:3037] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:04.375432 26250 raft_consensus.cc:504] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.376021 26250 leader_election.cc:288] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [CANDIDATE]: Term 1 election: Requested vote from peers d96e9408541a4e03aae7bc2b5f407a66 (127.6.215.194:44761), 761f7a98d94c43fab8913d220b439578 (127.6.215.195:44223)
I20260926 19:38:04.376547 26179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aaff1c67ec64640b420cc66ffce56d0" candidate_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "761f7a98d94c43fab8913d220b439578"
I20260926 19:38:04.376530 26049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aaff1c67ec64640b420cc66ffce56d0" candidate_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d96e9408541a4e03aae7bc2b5f407a66"
I20260926 19:38:04.376761 26049 raft_consensus.cc:3037] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:04.376748 26179 raft_consensus.cc:3037] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:04.380093 26049 raft_consensus.cc:2454] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb569a5082dc4a35b6a40aa8eb4b091e in term 1.
I20260926 19:38:04.380100 26179 raft_consensus.cc:2454] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb569a5082dc4a35b6a40aa8eb4b091e in term 1.
I20260926 19:38:04.380479 25853 leader_election.cc:302] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 761f7a98d94c43fab8913d220b439578, fb569a5082dc4a35b6a40aa8eb4b091e; no voters:
I20260926 19:38:04.380823 26250 raft_consensus.cc:2781] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:04.381338 26250 raft_consensus.cc:686] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 1 LEADER]: Becoming Leader. State: Replica: fb569a5082dc4a35b6a40aa8eb4b091e, State: Running, Role: LEADER
I20260926 19:38:04.381816 26250 consensus_queue.cc:237] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } }
I20260926 19:38:04.386283 25776 catalog_manager.cc:5495] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e reported cstate change: term changed from 0 to 1, leader changed from <none> to fb569a5082dc4a35b6a40aa8eb4b091e (127.6.215.193). New cstate: current_term: 1 leader_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb569a5082dc4a35b6a40aa8eb4b091e" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 39717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 } health_report { overall_health: UNKNOWN } } }
W20260926 19:38:04.411919 26095 tablet.cc:2369] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:38:04.425400 25964 tablet.cc:2369] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:04.444598 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.439130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.6.215.193:39717)
0926 19:38:04.439360 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.439365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.439385 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:38:04.439591 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.439594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.439686 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.439838 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:04.440228 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.441020 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.442374 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.442788 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.442925 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.443506 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:04.443993 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:04.444191 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.444337 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.444383 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 19:38:04.452637 26179 raft_consensus.cc:1261] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 1 FOLLOWER]: Refusing update from remote peer fb569a5082dc4a35b6a40aa8eb4b091e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:04.452690 26049 raft_consensus.cc:1261] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 1 FOLLOWER]: Refusing update from remote peer fb569a5082dc4a35b6a40aa8eb4b091e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:04.453660 26253 consensus_queue.cc:1035] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d96e9408541a4e03aae7bc2b5f407a66" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 44761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:04.454003 26250 consensus_queue.cc:1035] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [LEADER]: Connected to new peer: Peer: permanent_uuid: "761f7a98d94c43fab8913d220b439578" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 44223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:04.483126 25776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:04.483443 25776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:04.487182 25776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49af3319bb0f47fe9ac79f6c6b440ef0: Sending DeleteTablet for 3 replicas of tablet 2aaff1c67ec64640b420cc66ffce56d0
I20260926 19:38:04.488332 26029 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aaff1c67ec64640b420cc66ffce56d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:04 UTC) from {username='slave'} at 127.0.0.1:49308
I20260926 19:38:04.488390 25898 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aaff1c67ec64640b420cc66ffce56d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:04 UTC) from {username='slave'} at 127.0.0.1:52210
I20260926 19:38:04.488900 26159 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aaff1c67ec64640b420cc66ffce56d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:04 UTC) from {username='slave'} at 127.0.0.1:35766
I20260926 19:38:04.491143 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25836
I20260926 19:38:04.498646 26268 tablet_replica.cc:330] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: stopping tablet replica
I20260926 19:38:04.499235 26268 raft_consensus.cc:2229] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:04.499825 26268 raft_consensus.cc:2258] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:04.506327 26267 tablet_replica.cc:330] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: stopping tablet replica
I20260926 19:38:04.506814 26267 raft_consensus.cc:2229] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:38:04.507544 26267 raft_consensus.cc:2258] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:04.508654 26268 ts_tablet_manager.cc:1905] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:04.512389 26267 ts_tablet_manager.cc:1905] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:04.517282 26266 tablet_replica.cc:330] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: stopping tablet replica
I20260926 19:38:04.517557 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.492523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52222 (local address 127.6.215.193:39717)
0926 19:38:04.492783 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.492793 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.506641 (+ 13848us) server_negotiation.cc:407] Connection header received
0926 19:38:04.506872 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.506879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.506992 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.507118 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:04.509506 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.510754 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.513281 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.513975 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.514084 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.515591 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:04.515602 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:04.515624 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:04.516421 (+ 797us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:04.516489 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:04.516513 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:04.516621 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.516924 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.517040 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:38:04.517742 26266 raft_consensus.cc:2229] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:04.518576 26266 raft_consensus.cc:2258] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:04.519163 25958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:04.516716 26268 ts_tablet_manager.cc:1918] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:04.520684 26268 log.cc:1196] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/2aaff1c67ec64640b420cc66ffce56d0
I20260926 19:38:04.521347 26268 ts_tablet_manager.cc:1939] T 2aaff1c67ec64640b420cc66ffce56d0 P 761f7a98d94c43fab8913d220b439578: Deleting consensus metadata
I20260926 19:38:04.522423 26266 ts_tablet_manager.cc:1905] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:04.523350 25763 catalog_manager.cc:4841] TS 761f7a98d94c43fab8913d220b439578 (127.6.215.195:44223): tablet 2aaff1c67ec64640b420cc66ffce56d0 (table test-table [id=611821e117c144c6968aeddd5c394bfc]) successfully deleted
I20260926 19:38:04.528535 26266 ts_tablet_manager.cc:1918] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:04.528702 26266 log.cc:1196] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/2aaff1c67ec64640b420cc66ffce56d0
I20260926 19:38:04.529215 26266 ts_tablet_manager.cc:1939] T 2aaff1c67ec64640b420cc66ffce56d0 P d96e9408541a4e03aae7bc2b5f407a66: Deleting consensus metadata
I20260926 19:38:04.530754 25762 catalog_manager.cc:4841] TS d96e9408541a4e03aae7bc2b5f407a66 (127.6.215.194:44761): tablet 2aaff1c67ec64640b420cc66ffce56d0 (table test-table [id=611821e117c144c6968aeddd5c394bfc]) successfully deleted
I20260926 19:38:04.618515 26267 ts_tablet_manager.cc:1918] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:04.619201 26267 log.cc:1196] T 2aaff1c67ec64640b420cc66ffce56d0 P fb569a5082dc4a35b6a40aa8eb4b091e: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/2aaff1c67ec64640b420cc66ffce56d0
I20260926 19:38:04.619733 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25836
I20260926 19:38:04.639884 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25967
W20260926 19:38:04.640450 25761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.215.193:39717 (error 108)
I20260926 19:38:04.642859 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.642349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52226)
0926 19:38:04.642540 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.642693 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 19:38:04.643115 25761 catalog_manager.cc:4584] TS fb569a5082dc4a35b6a40aa8eb4b091e (127.6.215.193:39717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2aaff1c67ec64640b420cc66ffce56d0: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
I20260926 19:38:04.647922 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.641373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.6.215.194:44761)
0926 19:38:04.641582 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.641588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.641608 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:38:04.641670 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.641676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.641773 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.641884 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:04.642544 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.643579 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.644974 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.645380 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.645514 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.646615 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:04.646624 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:04.646637 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:04.647191 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:04.647237 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:04.647257 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:04.647391 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.647527 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.647630 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 19:38:04.648635 26089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:04.696722 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.696238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52228)
0926 19:38:04.696455 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.696567 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 19:38:04.697897 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.697494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52230)
0926 19:38:04.697677 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.697751 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 19:38:04.737171 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25967
I20260926 19:38:04.753844 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 26097
I20260926 19:38:04.758283 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.757852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52246)
0926 19:38:04.758032 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.758146 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 19:38:04.759402 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.758974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52248)
0926 19:38:04.759175 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.759263 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 19:38:04.760488 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.754607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.6.215.195:44223)
0926 19:38:04.754780 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:04.754785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:04.754804 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:38:04.754913 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:04.754917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:04.754997 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:04.755078 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:04.755434 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.756134 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.757390 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:04.757795 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:04.757932 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:04.759162 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:04.759173 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:04.759193 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:04.759891 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:04.759938 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:04.759958 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:04.760043 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:04.760176 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 19:38:04.760263 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:38:04.761145 26219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:04.840240 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.839619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52262)
0926 19:38:04.839897 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.840018 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 19:38:04.841465 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:04.841035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:39717 (local address 127.0.0.1:52264)
0926 19:38:04.841246 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:04.841333 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.193:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 19:38:04.850405 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 26097
I20260926 19:38:04.866890 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 25746
I20260926 19:38:04.867995 25807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:04.979490 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 25746
2026-09-26T19:38:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2839 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:05 dist-test-slave-phbh krb5kdc[26277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:05 dist-test-slave-phbh krb5kdc[26277](info): set up 2 sockets
Sep 26 19:38:05 dist-test-slave-phbh krb5kdc[26277](info): commencing operation
krb5kdc: starting...
W20260926 19:38:06.105216 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:06 dist-test-slave-phbh krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:06.355548 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42107
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:06.693116 26290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:06.693425 26290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:06.693521 26290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:06.700879 26290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:06.700987 26290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:06.701053 26290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:06.701102 26290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:06.701149 26290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:06.712055 26290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:06.713449 26290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:06.714653 26290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:06.726420 26296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:06.726503 26298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:06.726460 26295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:06.727603 26290 server_base.cc:1034] running on GCE node
I20260926 19:38:06.728489 26290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:06.729950 26290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:06.731195 26290 hybrid_clock.cc:648] HybridClock initialized: now 1790451486731150 us; error 55 us; skew 500 ppm
I20260926 19:38:06.731544 26290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:06 dist-test-slave-phbh krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451486, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:06.741150 26290 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:06.742745 26290 webserver.cc:460] Webserver started at http://127.6.215.254:42613/ using document root <none> and password file <none>
I20260926 19:38:06.743474 26290 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:06.743623 26290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:06.743944 26290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:06.746996 26290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "3fcb6a857d8a4afb8345ba8a34486e90"
format_stamp: "Formatted at 2026-09-26 19:38:06 on dist-test-slave-phbh"
server_key: "b0fb519dcd61d040bfffb3848e831cf7"
server_key_iv: "f2243949c0bd4f0a2195340f7467d783"
server_key_version: "encryptionkey@0"
I20260926 19:38:06.747776 26290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "3fcb6a857d8a4afb8345ba8a34486e90"
format_stamp: "Formatted at 2026-09-26 19:38:06 on dist-test-slave-phbh"
server_key: "b0fb519dcd61d040bfffb3848e831cf7"
server_key_iv: "f2243949c0bd4f0a2195340f7467d783"
server_key_version: "encryptionkey@0"
I20260926 19:38:06.752919 26290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260926 19:38:06.756197 26305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:06.757560 26290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:06.757745 26290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "3fcb6a857d8a4afb8345ba8a34486e90"
format_stamp: "Formatted at 2026-09-26 19:38:06 on dist-test-slave-phbh"
server_key: "b0fb519dcd61d040bfffb3848e831cf7"
server_key_iv: "f2243949c0bd4f0a2195340f7467d783"
server_key_version: "encryptionkey@0"
I20260926 19:38:06.757936 26290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:06.799924 26290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:06.804455 26290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:06.804759 26290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:06.823359 26290 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42107
I20260926 19:38:06.823379 26356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42107 every 8 connection(s)
I20260926 19:38:06.824873 26290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:06.828487 26357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:06.830436 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 26290
I20260926 19:38:06.830643 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:06.831202 7007 external_mini_cluster.cc:1428] Setting key 9ad17bb7e74bfa6a95d599aea4a936dd
I20260926 19:38:06.842649 26357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90: Bootstrap starting.
Sep 26 19:38:06 dist-test-slave-phbh krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:06.846688 26357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:06.847911 26357 log.cc:824] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:06.851227 26357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90: No bootstrap required, opened a new log
I20260926 19:38:06.854976 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:06.833148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.6.215.254:42107)
0926 19:38:06.833995 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:06.834010 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:06.834050 (+ 40us) server_negotiation.cc:407] Connection header received
0926 19:38:06.835651 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:06.835675 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:06.836053 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:06.836352 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:06.838247 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:06.839580 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:06.841561 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:06.842217 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:06.842352 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:06.847346 (+ 4994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:06.847353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:06.847368 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:06.847403 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:06.850791 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:06.851660 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:06.851667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:06.851672 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:06.851839 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:06.852295 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:06.852298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:06.852300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:06.852868 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:06.853110 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:06.853564 (+ 454us) server_negotiation.cc:299] Negotiation successful
0926 19:38:06.853962 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":422,"threads_started":1}
I20260926 19:38:06.859977 26357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42107 } }
I20260926 19:38:06.860356 26357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:06.860477 26357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fcb6a857d8a4afb8345ba8a34486e90, State: Initialized, Role: FOLLOWER
I20260926 19:38:06.861102 26357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42107 } }
I20260926 19:38:06.861339 26357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:06.861464 26357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:06.861639 26357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:06.864128 26357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42107 } }
I20260926 19:38:06.864605 26357 leader_election.cc:302] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fcb6a857d8a4afb8345ba8a34486e90; no voters:
I20260926 19:38:06.865207 26357 leader_election.cc:288] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:06.865381 26362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:06.866077 26362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [term 1 LEADER]: Becoming Leader. State: Replica: 3fcb6a857d8a4afb8345ba8a34486e90, State: Running, Role: LEADER
I20260926 19:38:06.866603 26362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42107 } }
I20260926 19:38:06.867233 26357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:06.869510 26363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42107 } } }
I20260926 19:38:06.869987 26363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:06.871781 26364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fcb6a857d8a4afb8345ba8a34486e90. Latest consensus state: current_term: 1 leader_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fcb6a857d8a4afb8345ba8a34486e90" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42107 } } }
I20260926 19:38:06.872140 26364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:06.872982 26371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:06.878103 26371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:06.887195 26371 catalog_manager.cc:1347] Generated new cluster ID: bf4e0f68c8dd4f45a039c6d41473335a
I20260926 19:38:06.887311 26371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:06.901108 26371 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:06.902395 26371 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:06.918964 26371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90: Generated new TSK 0
I20260926 19:38:06.919929 26371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.215.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
Entry for principal kudu/127.6.215.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab.
I20260926 19:38:06.998564 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42107
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:07.332116 26383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:07.332417 26383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:07.332516 26383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:07.339628 26383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:07.339737 26383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:07.339848 26383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:38:07.349792 26383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.193.keytab
--principal=kudu/127.6.215.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:07.351007 26383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:07.352349 26383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:07.365254 26391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:07.365381 26389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:07.365348 26388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:07.366277 26383 server_base.cc:1034] running on GCE node
I20260926 19:38:07.366928 26383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:07.367882 26383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:07.369073 26383 hybrid_clock.cc:648] HybridClock initialized: now 1790451487369024 us; error 49 us; skew 500 ppm
I20260926 19:38:07.369437 26383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:07 dist-test-slave-phbh krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451487, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:07.372731 26383 init.cc:377] Logged in from keytab as kudu/127.6.215.193@KRBTEST.COM (short username kudu)
I20260926 19:38:07.374110 26383 webserver.cc:460] Webserver started at http://127.6.215.193:32987/ using document root <none> and password file <none>
I20260926 19:38:07.374727 26383 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:07.374825 26383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:07.375072 26383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:07.378106 26383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "59480a9680884bc9b310e1bfa20ad056"
format_stamp: "Formatted at 2026-09-26 19:38:07 on dist-test-slave-phbh"
server_key: "418146d0c8fdd333f4255bd5bd6177cc"
server_key_iv: "3ac0bf1fb51da981055a21640421b909"
server_key_version: "encryptionkey@0"
I20260926 19:38:07.378831 26383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "59480a9680884bc9b310e1bfa20ad056"
format_stamp: "Formatted at 2026-09-26 19:38:07 on dist-test-slave-phbh"
server_key: "418146d0c8fdd333f4255bd5bd6177cc"
server_key_iv: "3ac0bf1fb51da981055a21640421b909"
server_key_version: "encryptionkey@0"
I20260926 19:38:07.383865 26383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 19:38:07.387223 26398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:07.388489 26383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:38:07.388676 26383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "59480a9680884bc9b310e1bfa20ad056"
format_stamp: "Formatted at 2026-09-26 19:38:07 on dist-test-slave-phbh"
server_key: "418146d0c8fdd333f4255bd5bd6177cc"
server_key_iv: "3ac0bf1fb51da981055a21640421b909"
server_key_version: "encryptionkey@0"
I20260926 19:38:07.388954 26383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:07.421952 26383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:07.426630 26383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:07.426952 26383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:07.428000 26383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:07.429467 26383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:07.429557 26383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:07.429746 26383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:07.429816 26383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:07.466497 26383 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:43575
I20260926 19:38:07.466536 26510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:43575 every 8 connection(s)
I20260926 19:38:07.467983 26383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:38:07.474269 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 26383
I20260926 19:38:07.474473 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:38:07.474900 7007 external_mini_cluster.cc:1428] Setting key 6bab6cfae2d7f919de0f71ff974b5de6
Sep 26 19:38:07 dist-test-slave-phbh krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451487, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.193@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:07.488974 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:07.470696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:55603 (local address 127.6.215.254:42107)
0926 19:38:07.471152 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:07.471158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:07.472146 (+ 988us) server_negotiation.cc:407] Connection header received
0926 19:38:07.474136 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:07.474142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:07.474237 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:07.474360 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:07.476003 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:07.476764 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:07.478385 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:07.478786 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:07.478922 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:07.482719 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:07.482724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:07.482727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:07.482759 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:07.485555 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:07.486502 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:07.486507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:07.486510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:07.486647 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:07.487157 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:07.487161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:07.487163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:07.487432 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:07.487555 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:07.488463 (+ 908us) server_negotiation.cc:299] Negotiation successful
0926 19:38:07.488648 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":211,"threads_started":1}
I20260926 19:38:07.490738 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:07.471311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42107 (local address 127.6.215.193:55603)
0926 19:38:07.471974 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:07.472013 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:07.473865 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:07.474661 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:07.474669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:07.475258 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:07.475826 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:07.475845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:07.477001 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:07.477006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:07.478235 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:07.478246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:07.479032 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:07.479042 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:07.479271 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:07.479970 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:07.479992 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:07.482468 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:07.485708 (+ 3240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:07.485720 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:07.485740 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:07.486380 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:07.486767 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:07.486775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:07.486781 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:07.487032 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:07.487566 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:07.487577 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:07.488047 (+ 470us) client_negotiation.cc:769] Sending connection context
0926 19:38:07.488381 (+ 334us) client_negotiation.cc:240] Negotiation successful
0926 19:38:07.488994 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":263,"threads_started":1}
I20260926 19:38:07.492496 26511 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42107
I20260926 19:38:07.492897 26511 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:07.493701 26511 heartbeater.cc:511] Master 127.6.215.254:42107 requested a full tablet report, sending...
I20260926 19:38:07.495908 26322 ts_manager.cc:194] Registered new tserver with Master: 59480a9680884bc9b310e1bfa20ad056 (127.6.215.193:43575)
I20260926 19:38:07.497474 26322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.193@KRBTEST.COM'} at 127.6.215.193:55603
WARNING: no policy specified for kudu/127.6.215.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
Entry for principal kudu/127.6.215.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab.
I20260926 19:38:07.535820 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42107
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:07.870699 26517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:07.871011 26517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:07.871177 26517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:07.878615 26517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:07.878769 26517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:07.878942 26517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:38:07.888821 26517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.194.keytab
--principal=kudu/127.6.215.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:07.890126 26517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:07.891431 26517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:07.905187 26523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:07.905222 26525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:07.905191 26522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:07.905982 26517 server_base.cc:1034] running on GCE node
I20260926 19:38:07.906596 26517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:07.907524 26517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:07.908753 26517 hybrid_clock.cc:648] HybridClock initialized: now 1790451487908709 us; error 55 us; skew 500 ppm
I20260926 19:38:07.909128 26517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:07 dist-test-slave-phbh krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451487, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:07.912144 26517 init.cc:377] Logged in from keytab as kudu/127.6.215.194@KRBTEST.COM (short username kudu)
I20260926 19:38:07.913686 26517 webserver.cc:460] Webserver started at http://127.6.215.194:38809/ using document root <none> and password file <none>
I20260926 19:38:07.914359 26517 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:07.914503 26517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:07.914813 26517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:07.917650 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "87b3d5aaef5e4b2086d94a91644b108f"
format_stamp: "Formatted at 2026-09-26 19:38:07 on dist-test-slave-phbh"
server_key: "5ab97d6ec0822edcd5d0fd77a4115937"
server_key_iv: "1c14567045ce06e42e61c5331865e6bf"
server_key_version: "encryptionkey@0"
I20260926 19:38:07.918404 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "87b3d5aaef5e4b2086d94a91644b108f"
format_stamp: "Formatted at 2026-09-26 19:38:07 on dist-test-slave-phbh"
server_key: "5ab97d6ec0822edcd5d0fd77a4115937"
server_key_iv: "1c14567045ce06e42e61c5331865e6bf"
server_key_version: "encryptionkey@0"
I20260926 19:38:07.923547 26517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:07.927052 26532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:07.928282 26517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:38:07.928448 26517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "87b3d5aaef5e4b2086d94a91644b108f"
format_stamp: "Formatted at 2026-09-26 19:38:07 on dist-test-slave-phbh"
server_key: "5ab97d6ec0822edcd5d0fd77a4115937"
server_key_iv: "1c14567045ce06e42e61c5331865e6bf"
server_key_version: "encryptionkey@0"
I20260926 19:38:07.928651 26517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:07.977654 26517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:07.982198 26517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:07.982503 26517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:07.983498 26517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:07.984905 26517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:07.985002 26517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:07.985136 26517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:07.985211 26517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:08.018469 26517 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:43225
I20260926 19:38:08.018527 26644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:43225 every 8 connection(s)
I20260926 19:38:08.019780 26517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:38:08.020668 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 26517
I20260926 19:38:08.020871 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:38:08.021445 7007 external_mini_cluster.cc:1428] Setting key 70935744eaa804f6fffad75d8e3b731d
Sep 26 19:38:08 dist-test-slave-phbh krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451487, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.194@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:08.042514 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.022615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:53615 (local address 127.6.215.254:42107)
0926 19:38:08.023099 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.023105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.024346 (+ 1241us) server_negotiation.cc:407] Connection header received
0926 19:38:08.026259 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.026267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.026354 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.026480 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:08.028099 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.029315 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.030869 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.031525 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.031674 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.036642 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:08.036649 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:08.036655 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:08.036693 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:08.039410 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.040140 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.040146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.040150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.040281 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.040705 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.040711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.040714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.041043 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:08.041200 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.041899 (+ 699us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.042129 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":234,"threads_started":1}
I20260926 19:38:08.043486 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.023321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42107 (local address 127.6.215.194:53615)
0926 19:38:08.024169 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:08.024220 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:08.025987 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:08.026813 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:08.026822 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:08.027346 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:08.027914 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.027931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.029461 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.029465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.030718 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.030730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.031800 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.031807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.032148 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.033191 (+ 1043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:08.033228 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:08.036413 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:08.039563 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:08.039572 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:08.039589 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:08.040001 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:08.040394 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:08.040399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:08.040403 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:08.040586 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:08.041184 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:08.041192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:08.041537 (+ 345us) client_negotiation.cc:769] Sending connection context
0926 19:38:08.041818 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 19:38:08.042352 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":325,"threads_started":1}
I20260926 19:38:08.044955 26645 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42107
I20260926 19:38:08.045327 26645 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:08.046252 26645 heartbeater.cc:511] Master 127.6.215.254:42107 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.215.195@KRBTEST.COM; defaulting to no policy
I20260926 19:38:08.047906 26322 ts_manager.cc:194] Registered new tserver with Master: 87b3d5aaef5e4b2086d94a91644b108f (127.6.215.194:43225)
I20260926 19:38:08.048759 26322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.194@KRBTEST.COM'} at 127.6.215.194:53615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
Entry for principal kudu/127.6.215.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab.
I20260926 19:38:08.080863 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42107
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:08.405404 26651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:08.405679 26651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:08.405772 26651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:08.413049 26651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:08.413163 26651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:08.413276 26651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:38:08.422991 26651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.195.keytab
--principal=kudu/127.6.215.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:42107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:08.424218 26651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:08.425483 26651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:08.439298 26656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:08.439229 26659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:08.439265 26657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:08.440256 26651 server_base.cc:1034] running on GCE node
I20260926 19:38:08.440928 26651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:08.441856 26651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:08.443068 26651 hybrid_clock.cc:648] HybridClock initialized: now 1790451488443001 us; error 57 us; skew 500 ppm
I20260926 19:38:08.443404 26651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:08 dist-test-slave-phbh krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451488, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:08.446501 26651 init.cc:377] Logged in from keytab as kudu/127.6.215.195@KRBTEST.COM (short username kudu)
I20260926 19:38:08.447949 26651 webserver.cc:460] Webserver started at http://127.6.215.195:35325/ using document root <none> and password file <none>
I20260926 19:38:08.448640 26651 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:08.448827 26651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:08.449163 26651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:08.452680 26651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "930468a89b7c45b0b78fc85c80778394"
format_stamp: "Formatted at 2026-09-26 19:38:08 on dist-test-slave-phbh"
server_key: "6069fb3d5cc16faba1124f3d085d04a9"
server_key_iv: "732e4b723a6fe9a0b5aa21b6f8697346"
server_key_version: "encryptionkey@0"
I20260926 19:38:08.453635 26651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "930468a89b7c45b0b78fc85c80778394"
format_stamp: "Formatted at 2026-09-26 19:38:08 on dist-test-slave-phbh"
server_key: "6069fb3d5cc16faba1124f3d085d04a9"
server_key_iv: "732e4b723a6fe9a0b5aa21b6f8697346"
server_key_version: "encryptionkey@0"
I20260926 19:38:08.458979 26651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:08.462865 26666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:08.464345 26651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:38:08.464648 26651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "930468a89b7c45b0b78fc85c80778394"
format_stamp: "Formatted at 2026-09-26 19:38:08 on dist-test-slave-phbh"
server_key: "6069fb3d5cc16faba1124f3d085d04a9"
server_key_iv: "732e4b723a6fe9a0b5aa21b6f8697346"
server_key_version: "encryptionkey@0"
I20260926 19:38:08.464913 26651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:08.486579 26651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:08.490988 26651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:08.491312 26651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:08.492385 26651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:08.494017 26651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:08.494124 26651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:08.494268 26651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:08.494338 26651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:08.500873 26511 heartbeater.cc:503] Master 127.6.215.254:42107 was elected leader, sending a full tablet report...
I20260926 19:38:08.534296 26651 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:41663
I20260926 19:38:08.534358 26778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:41663 every 8 connection(s)
I20260926 19:38:08.535627 26651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:38:08.545068 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 26651
I20260926 19:38:08.545295 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:38:08.545742 7007 external_mini_cluster.cc:1428] Setting key 4a43d11776eb45818b38651722772e83
Sep 26 19:38:08 dist-test-slave-phbh krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451488, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.195@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:08.556083 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.538244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:54849 (local address 127.6.215.254:42107)
0926 19:38:08.538434 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.538439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.539594 (+ 1155us) server_negotiation.cc:407] Connection header received
0926 19:38:08.541545 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.541550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.541634 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.541727 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:08.543234 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.543952 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.545910 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.546470 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.546612 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.550597 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:08.550605 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:08.550610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:08.550644 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:08.553112 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.553841 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.553846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.553848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.553963 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.554395 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.554400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.554405 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.554679 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:08.554840 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.555517 (+ 677us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.555743 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:38:08.557216 26781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.538703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:42107 (local address 127.6.215.195:54849)
0926 19:38:08.539420 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:08.539465 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:08.541289 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:08.541939 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:08.541947 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:08.542507 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:08.543073 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.543090 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.544079 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.544082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.545754 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.545766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.546722 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.546729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.547014 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.547823 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:08.547862 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:08.550376 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:08.553249 (+ 2873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:08.553258 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:08.553276 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:08.553705 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:08.554077 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:08.554082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:08.554086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:08.554273 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:08.554812 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:08.554820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:08.555162 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 19:38:08.555419 (+ 257us) client_negotiation.cc:240] Negotiation successful
0926 19:38:08.555970 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":271,"threads_started":1}
I20260926 19:38:08.558593 26779 heartbeater.cc:347] Connected to a master server at 127.6.215.254:42107
I20260926 19:38:08.558948 26779 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:08.559866 26779 heartbeater.cc:511] Master 127.6.215.254:42107 requested a full tablet report, sending...
I20260926 19:38:08.561580 26322 ts_manager.cc:194] Registered new tserver with Master: 930468a89b7c45b0b78fc85c80778394 (127.6.215.195:41663)
I20260926 19:38:08.562541 26322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.215.195@KRBTEST.COM'} at 127.6.215.195:54849
I20260926 19:38:08.573849 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:38:08.590279 26647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.578000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.6.215.254:42107)
0926 19:38:08.578270 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.578276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.578521 (+ 245us) server_negotiation.cc:407] Connection header received
0926 19:38:08.578651 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.578654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.578718 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.578832 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:08.580171 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.581008 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.582328 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.582730 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.582827 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.585412 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:08.585417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:08.585420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:08.585451 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:08.587807 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.588412 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.588416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.588418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.588596 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.588934 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.588937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.588939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.589238 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:08.589431 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.589725 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.589879 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260926 19:38:08.594421 26322 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:38:08.601229 26322 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:38:08.627475 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.614131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:43575 (local address 127.0.0.1:40408)
0926 19:38:08.617493 (+ 3362us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:08.617547 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:08.617870 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:08.618990 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:08.618998 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:08.619033 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:08.619466 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.619479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.622544 (+ 3065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.622550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.624521 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.624536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.626589 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.626593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.626742 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.626774 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 19:38:08.626915 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 19:38:08.627105 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3025,"thread_start_us":4091,"threads_started":1}
I20260926 19:38:08.627471 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.614406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.6.215.193:43575)
0926 19:38:08.615170 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.615193 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.617681 (+ 2488us) server_negotiation.cc:407] Connection header received
0926 19:38:08.618156 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.618168 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.618476 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.618642 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:08.621147 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.622401 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.624720 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.626201 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.626291 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.626453 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.626958 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.627124 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":687,"threads_started":1}
I20260926 19:38:08.630088 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.614892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:43225 (local address 127.0.0.1:58140)
0926 19:38:08.618555 (+ 3663us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:08.618578 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:08.618717 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:08.620694 (+ 1977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:08.620698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:08.620721 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:08.621249 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.621265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.623653 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.623658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.625876 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.625892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.628326 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.628336 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.628609 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.628664 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 19:38:08.629403 (+ 739us) client_negotiation.cc:240] Negotiation successful
0926 19:38:08.629610 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3510,"thread_start_us":2087,"threads_started":1}
I20260926 19:38:08.631032 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.615066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.6.215.194:43225)
0926 19:38:08.620035 (+ 4969us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.620046 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.620074 (+ 28us) server_negotiation.cc:407] Connection header received
0926 19:38:08.620211 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.620222 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.620495 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.620724 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:08.621444 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.623488 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.626075 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.628036 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.630355 (+ 2319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.630575 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.630652 (+ 77us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.630804 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4777,"thread_start_us":4664,"threads_started":1}
I20260926 19:38:08.634284 26446 tablet_service.cc:1467] Processing CreateTablet for tablet 91e213ad73b2430c99af1a579f0b12e9 (DEFAULT_TABLE table=test-table [id=8e63f1dafaf84617a5c22845bd54f1b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:08.634663 26580 tablet_service.cc:1467] Processing CreateTablet for tablet 91e213ad73b2430c99af1a579f0b12e9 (DEFAULT_TABLE table=test-table [id=8e63f1dafaf84617a5c22845bd54f1b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:08.635672 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.619740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:41663 (local address 127.0.0.1:60330)
0926 19:38:08.628004 (+ 8264us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:08.628020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:08.628162 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:08.628719 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:08.628723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:08.628747 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:08.628962 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.628973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.631523 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.631528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.633378 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.633391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.634886 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.634892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.635080 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.635128 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 19:38:08.635241 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 19:38:08.635387 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8190,"thread_start_us":10504,"threads_started":1}
I20260926 19:38:08.635955 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.618959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60330 (local address 127.6.215.195:41663)
0926 19:38:08.619839 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.619850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.628137 (+ 8287us) server_negotiation.cc:407] Connection header received
0926 19:38:08.628343 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.628350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.628534 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.628734 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:08.629096 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.631337 (+ 2241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.633539 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.634723 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.634876 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.635184 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.635305 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.635457 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":456,"threads_started":1}
I20260926 19:38:08.636233 26580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91e213ad73b2430c99af1a579f0b12e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:08.636409 26446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91e213ad73b2430c99af1a579f0b12e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:08.638615 26714 tablet_service.cc:1467] Processing CreateTablet for tablet 91e213ad73b2430c99af1a579f0b12e9 (DEFAULT_TABLE table=test-table [id=8e63f1dafaf84617a5c22845bd54f1b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:08.639616 26714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91e213ad73b2430c99af1a579f0b12e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:08.650164 26797 tablet_bootstrap.cc:492] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Bootstrap starting.
I20260926 19:38:08.653493 26797 tablet_bootstrap.cc:654] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:08.653489 26798 tablet_bootstrap.cc:492] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Bootstrap starting.
I20260926 19:38:08.654888 26797 log.cc:824] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:08.657011 26798 tablet_bootstrap.cc:654] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:08.658226 26798 log.cc:824] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:08.658864 26797 tablet_bootstrap.cc:492] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: No bootstrap required, opened a new log
I20260926 19:38:08.659160 26797 ts_tablet_manager.cc:1397] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 19:38:08.656320 26799 tablet_bootstrap.cc:492] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Bootstrap starting.
I20260926 19:38:08.661245 26798 tablet_bootstrap.cc:492] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: No bootstrap required, opened a new log
I20260926 19:38:08.661495 26798 ts_tablet_manager.cc:1397] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260926 19:38:08.663872 26799 tablet_bootstrap.cc:654] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:08.664860 26799 log.cc:824] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:08.667330 26799 tablet_bootstrap.cc:492] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: No bootstrap required, opened a new log
I20260926 19:38:08.667650 26799 ts_tablet_manager.cc:1397] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.006s
I20260926 19:38:08.669797 26797 raft_consensus.cc:348] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.670245 26797 raft_consensus.cc:374] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:08.670343 26797 raft_consensus.cc:729] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 930468a89b7c45b0b78fc85c80778394, State: Initialized, Role: FOLLOWER
I20260926 19:38:08.670959 26797 consensus_queue.cc:260] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.672052 26779 heartbeater.cc:503] Master 127.6.215.254:42107 was elected leader, sending a full tablet report...
I20260926 19:38:08.672286 26798 raft_consensus.cc:348] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.672734 26798 raft_consensus.cc:374] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:08.672829 26797 ts_tablet_manager.cc:1428] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260926 19:38:08.672909 26798 raft_consensus.cc:729] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87b3d5aaef5e4b2086d94a91644b108f, State: Initialized, Role: FOLLOWER
I20260926 19:38:08.673480 26798 consensus_queue.cc:260] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.674561 26645 heartbeater.cc:503] Master 127.6.215.254:42107 was elected leader, sending a full tablet report...
I20260926 19:38:08.676590 26798 ts_tablet_manager.cc:1428] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260926 19:38:08.679736 26799 raft_consensus.cc:348] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.680116 26799 raft_consensus.cc:374] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:08.680228 26799 raft_consensus.cc:729] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59480a9680884bc9b310e1bfa20ad056, State: Initialized, Role: FOLLOWER
I20260926 19:38:08.680908 26799 consensus_queue.cc:260] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.682312 26799 ts_tablet_manager.cc:1428] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260926 19:38:08.685670 26804 raft_consensus.cc:482] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:38:08.685909 26804 raft_consensus.cc:504] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.687333 26804 leader_election.cc:288] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59480a9680884bc9b310e1bfa20ad056 (127.6.215.193:43575), 930468a89b7c45b0b78fc85c80778394 (127.6.215.195:41663)
I20260926 19:38:08.695024 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.687682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:38781 (local address 127.6.215.193:43575)
0926 19:38:08.687902 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.687908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.688185 (+ 277us) server_negotiation.cc:407] Connection header received
0926 19:38:08.688391 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.688399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.688497 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.688586 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:08.689297 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.690363 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.692872 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.694134 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.694298 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.694372 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.694678 (+ 306us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.694772 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:38:08.695231 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.687483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:43575 (local address 127.6.215.194:38781)
0926 19:38:08.688045 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:08.688085 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:08.688235 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:08.688670 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:08.688675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:08.688701 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:08.689152 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.689172 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.690484 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.690487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.692687 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.692701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.694321 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.694326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.694504 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.694545 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 19:38:08.694646 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 19:38:08.694777 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":1,"thread_start_us":297,"threads_started":1}
I20260926 19:38:08.695250 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.688370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:39861 (local address 127.6.215.195:41663)
0926 19:38:08.688578 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.688583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.688966 (+ 383us) server_negotiation.cc:407] Connection header received
0926 19:38:08.689127 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.689131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.689191 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.689298 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:08.689777 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.690739 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.693116 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.694346 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.694487 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.694533 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.694888 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.695012 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:38:08.695271 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.688174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:41663 (local address 127.6.215.194:39861)
0926 19:38:08.688856 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:08.688878 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:08.689004 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:08.689319 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:08.689321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:08.689341 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:08.689611 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.689617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.690858 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.690862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.692955 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:08.692967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.694534 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.694539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:08.694736 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.694765 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 19:38:08.694844 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 19:38:08.694970 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":298,"threads_started":1}
I20260926 19:38:08.696498 26734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e213ad73b2430c99af1a579f0b12e9" candidate_uuid: "87b3d5aaef5e4b2086d94a91644b108f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "930468a89b7c45b0b78fc85c80778394" is_pre_election: true
I20260926 19:38:08.696486 26466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e213ad73b2430c99af1a579f0b12e9" candidate_uuid: "87b3d5aaef5e4b2086d94a91644b108f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59480a9680884bc9b310e1bfa20ad056" is_pre_election: true
I20260926 19:38:08.697096 26466 raft_consensus.cc:2454] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87b3d5aaef5e4b2086d94a91644b108f in term 0.
I20260926 19:38:08.697083 26734 raft_consensus.cc:2454] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87b3d5aaef5e4b2086d94a91644b108f in term 0.
I20260926 19:38:08.697764 26535 leader_election.cc:302] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87b3d5aaef5e4b2086d94a91644b108f, 930468a89b7c45b0b78fc85c80778394; no voters:
I20260926 19:38:08.698148 26804 raft_consensus.cc:2781] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:38:08.698312 26804 raft_consensus.cc:482] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:38:08.698421 26804 raft_consensus.cc:3037] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:08.700870 26804 raft_consensus.cc:504] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.701421 26804 leader_election.cc:288] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [CANDIDATE]: Term 1 election: Requested vote from peers 59480a9680884bc9b310e1bfa20ad056 (127.6.215.193:43575), 930468a89b7c45b0b78fc85c80778394 (127.6.215.195:41663)
I20260926 19:38:08.701829 26466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e213ad73b2430c99af1a579f0b12e9" candidate_uuid: "87b3d5aaef5e4b2086d94a91644b108f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59480a9680884bc9b310e1bfa20ad056"
I20260926 19:38:08.702013 26466 raft_consensus.cc:3037] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:08.701997 26734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e213ad73b2430c99af1a579f0b12e9" candidate_uuid: "87b3d5aaef5e4b2086d94a91644b108f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "930468a89b7c45b0b78fc85c80778394"
I20260926 19:38:08.702208 26734 raft_consensus.cc:3037] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:08.704145 26466 raft_consensus.cc:2454] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87b3d5aaef5e4b2086d94a91644b108f in term 1.
I20260926 19:38:08.704397 26734 raft_consensus.cc:2454] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87b3d5aaef5e4b2086d94a91644b108f in term 1.
I20260926 19:38:08.704521 26535 leader_election.cc:302] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59480a9680884bc9b310e1bfa20ad056, 87b3d5aaef5e4b2086d94a91644b108f; no voters:
I20260926 19:38:08.704818 26804 raft_consensus.cc:2781] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:08.705241 26804 raft_consensus.cc:686] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 1 LEADER]: Becoming Leader. State: Replica: 87b3d5aaef5e4b2086d94a91644b108f, State: Running, Role: LEADER
I20260926 19:38:08.705684 26804 consensus_queue.cc:237] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } }
I20260926 19:38:08.710232 26322 catalog_manager.cc:5495] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f reported cstate change: term changed from 0 to 1, leader changed from <none> to 87b3d5aaef5e4b2086d94a91644b108f (127.6.215.194). New cstate: current_term: 1 leader_uuid: "87b3d5aaef5e4b2086d94a91644b108f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87b3d5aaef5e4b2086d94a91644b108f" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 43225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 } health_report { overall_health: UNKNOWN } } }
W20260926 19:38:08.720417 26512 tablet.cc:2369] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:08.727880 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.722479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.6.215.194:43225)
0926 19:38:08.722670 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.722678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.722766 (+ 88us) server_negotiation.cc:407] Connection header received
0926 19:38:08.722997 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.723001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.723076 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.723286 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:08.723662 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.724381 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.725795 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.726204 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.726335 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.726840 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:08.727308 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:08.727483 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.727533 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.727611 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:38:08.736164 26734 raft_consensus.cc:1261] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 1 FOLLOWER]: Refusing update from remote peer 87b3d5aaef5e4b2086d94a91644b108f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:08.736166 26466 raft_consensus.cc:1261] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 1 FOLLOWER]: Refusing update from remote peer 87b3d5aaef5e4b2086d94a91644b108f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:08.737227 26808 consensus_queue.cc:1035] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [LEADER]: Connected to new peer: Peer: permanent_uuid: "930468a89b7c45b0b78fc85c80778394" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:08.737583 26804 consensus_queue.cc:1035] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [LEADER]: Connected to new peer: Peer: permanent_uuid: "59480a9680884bc9b310e1bfa20ad056" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 43575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:08.747942 26811 mvcc.cc:205] Tried to move back new op lower bound from 7333689297853001728 to 7333689297742462976. Current Snapshot: MvccSnapshot[applied={T|T < 7333689297853001728}]
I20260926 19:38:08.748651 26812 mvcc.cc:205] Tried to move back new op lower bound from 7333689297853001728 to 7333689297742462976. Current Snapshot: MvccSnapshot[applied={T|T < 7333689297853001728}]
I20260926 19:38:08.754473 26813 mvcc.cc:205] Tried to move back new op lower bound from 7333689297853001728 to 7333689297742462976. Current Snapshot: MvccSnapshot[applied={T|T < 7333689297853001728}]
I20260926 19:38:08.761189 26322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:08.761490 26322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:08.765252 26322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fcb6a857d8a4afb8345ba8a34486e90: Sending DeleteTablet for 3 replicas of tablet 91e213ad73b2430c99af1a579f0b12e9
I20260926 19:38:08.766464 26714 tablet_service.cc:1514] Processing DeleteTablet for tablet 91e213ad73b2430c99af1a579f0b12e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:08 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:60330
I20260926 19:38:08.766469 26580 tablet_service.cc:1514] Processing DeleteTablet for tablet 91e213ad73b2430c99af1a579f0b12e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:08 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:58140
I20260926 19:38:08.766490 26446 tablet_service.cc:1514] Processing DeleteTablet for tablet 91e213ad73b2430c99af1a579f0b12e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:08 UTC) from {username='kudu', principal='kudu/127.6.215.254@KRBTEST.COM'} at 127.0.0.1:40408
I20260926 19:38:08.769392 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 26383
I20260926 19:38:08.778961 26821 tablet_replica.cc:330] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: stopping tablet replica
I20260926 19:38:08.779547 26821 raft_consensus.cc:2229] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:38:08.780366 26821 raft_consensus.cc:2258] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:08.782928 26821 ts_tablet_manager.cc:1905] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:08.791955 26821 ts_tablet_manager.cc:1918] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:08.792744 26822 tablet_replica.cc:330] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: stopping tablet replica
I20260926 19:38:08.792847 26821 log.cc:1196] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/91e213ad73b2430c99af1a579f0b12e9
I20260926 19:38:08.793300 26822 raft_consensus.cc:2229] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:08.793510 26821 ts_tablet_manager.cc:1939] T 91e213ad73b2430c99af1a579f0b12e9 P 87b3d5aaef5e4b2086d94a91644b108f: Deleting consensus metadata
I20260926 19:38:08.793949 26822 raft_consensus.cc:2258] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:08.795323 26309 catalog_manager.cc:4841] TS 87b3d5aaef5e4b2086d94a91644b108f (127.6.215.194:43225): tablet 91e213ad73b2430c99af1a579f0b12e9 (table test-table [id=8e63f1dafaf84617a5c22845bd54f1b2]) successfully deleted
I20260926 19:38:08.796380 26822 ts_tablet_manager.cc:1905] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:08.796429 26820 tablet_replica.cc:330] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: stopping tablet replica
I20260926 19:38:08.796891 26820 raft_consensus.cc:2229] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:08.797413 26820 raft_consensus.cc:2258] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:08.799160 26820 ts_tablet_manager.cc:1905] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:08.803953 26822 ts_tablet_manager.cc:1918] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:08.804203 26822 log.cc:1196] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/91e213ad73b2430c99af1a579f0b12e9
I20260926 19:38:08.804834 26822 ts_tablet_manager.cc:1939] T 91e213ad73b2430c99af1a579f0b12e9 P 59480a9680884bc9b310e1bfa20ad056: Deleting consensus metadata
Sep 26 19:38:08 dist-test-slave-phbh krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.193@KRBTEST.COM
I20260926 19:38:08.806593 26820 ts_tablet_manager.cc:1918] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:08.806787 26820 log.cc:1196] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/91e213ad73b2430c99af1a579f0b12e9
I20260926 19:38:08.806746 26308 catalog_manager.cc:4841] TS 59480a9680884bc9b310e1bfa20ad056 (127.6.215.193:43575): tablet 91e213ad73b2430c99af1a579f0b12e9 (table test-table [id=8e63f1dafaf84617a5c22845bd54f1b2]) successfully deleted
I20260926 19:38:08.807343 26820 ts_tablet_manager.cc:1939] T 91e213ad73b2430c99af1a579f0b12e9 P 930468a89b7c45b0b78fc85c80778394: Deleting consensus metadata
I20260926 19:38:08.808995 26308 catalog_manager.cc:4841] TS 930468a89b7c45b0b78fc85c80778394 (127.6.215.195:41663): tablet 91e213ad73b2430c99af1a579f0b12e9 (table test-table [id=8e63f1dafaf84617a5c22845bd54f1b2]) successfully deleted
I20260926 19:38:08.813756 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.770998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40412 (local address 127.6.215.193:43575)
0926 19:38:08.771230 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.771239 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.795762 (+ 24523us) server_negotiation.cc:407] Connection header received
0926 19:38:08.796004 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.796011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.796094 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.796199 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:08.798149 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.799297 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.801200 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.801895 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.801993 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.807659 (+ 5666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:08.807670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:08.807684 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:08.807720 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:08.811021 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.811710 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.811717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.811719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.811842 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.812219 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.812228 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.812230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.812550 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:08.812902 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.813191 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.813403 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 19:38:08.814486 26506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:08.903992 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 26383
I20260926 19:38:08.927574 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 26517
Sep 26 19:38:08 dist-test-slave-phbh krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.194@KRBTEST.COM
I20260926 19:38:08.945199 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:08.928345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.6.215.194:43225)
0926 19:38:08.928560 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:08.928566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:08.928591 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:38:08.928917 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:08.928923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:08.929005 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:08.929085 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:08.930823 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.931608 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.933533 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:08.933990 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:08.934102 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:08.939137 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:08.939151 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:08.939167 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:08.939205 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:08.942241 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.942965 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.942972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.942974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.943145 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:08.943564 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:08.943568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:08.943570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:08.943926 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:08.944278 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:08.944549 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 19:38:08.944836 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 19:38:08.945971 26640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:09.058044 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 26517
I20260926 19:38:09.077423 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 26651
Sep 26 19:38:09 dist-test-slave-phbh krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.195@KRBTEST.COM
I20260926 19:38:09.092806 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:09.078177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.6.215.195:41663)
0926 19:38:09.078397 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:09.078405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:09.078425 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:38:09.078632 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:09.078636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:09.078699 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:09.078812 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:09.080209 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:09.081025 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:09.082348 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:09.082764 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:09.082900 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:09.087280 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:09.087294 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:09.087310 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:09.087344 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:09.090003 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:09.090697 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:09.090704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:09.090706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:09.090838 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:09.091209 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:09.091216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:09.091219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:09.091569 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:09.091881 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:09.092201 (+ 320us) server_negotiation.cc:299] Negotiation successful
0926 19:38:09.092420 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 19:38:09.093499 26774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:09.185083 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 26651
I20260926 19:38:09.203707 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 26290
I20260926 19:38:09.204917 26352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:09.313933 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 26290
2026-09-26T19:38:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4335 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T19:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:09Z Disabled control of system clock
I20260926 19:38:09.391742 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46213
--webserver_interface=127.6.215.254
--webserver_port=0
--builtin_ntp_servers=127.6.215.212:38997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:46213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 19:38:09.722122 26831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:09.722409 26831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:09.722491 26831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:09.729748 26831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:09.729862 26831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:09.729900 26831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:09.729950 26831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:09.729995 26831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:09.739318 26831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:46213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:09.740620 26831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:09.742498 26831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:09.754208 26836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:09.754252 26839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:09.754148 26837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:09.754935 26831 server_base.cc:1034] running on GCE node
I20260926 19:38:09.755818 26831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:09.757289 26831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:09.758481 26831 hybrid_clock.cc:648] HybridClock initialized: now 1790451489758412 us; error 61 us; skew 500 ppm
I20260926 19:38:09.758831 26831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:09.761204 26831 webserver.cc:460] Webserver started at http://127.6.215.254:45135/ using document root <none> and password file <none>
I20260926 19:38:09.761884 26831 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:09.761968 26831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:09.762303 26831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:09.765195 26831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "c1215cfcb6fc4539929e825d52728f97"
format_stamp: "Formatted at 2026-09-26 19:38:09 on dist-test-slave-phbh"
server_key: "923482ba86886f1466fe70c1b15f19d3"
server_key_iv: "48d43b54eb8fc7285b8b29b5df6f9162"
server_key_version: "encryptionkey@0"
I20260926 19:38:09.766012 26831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "c1215cfcb6fc4539929e825d52728f97"
format_stamp: "Formatted at 2026-09-26 19:38:09 on dist-test-slave-phbh"
server_key: "923482ba86886f1466fe70c1b15f19d3"
server_key_iv: "48d43b54eb8fc7285b8b29b5df6f9162"
server_key_version: "encryptionkey@0"
I20260926 19:38:09.771344 26831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:09.774960 26845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:09.776546 26831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:38:09.776762 26831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "c1215cfcb6fc4539929e825d52728f97"
format_stamp: "Formatted at 2026-09-26 19:38:09 on dist-test-slave-phbh"
server_key: "923482ba86886f1466fe70c1b15f19d3"
server_key_iv: "48d43b54eb8fc7285b8b29b5df6f9162"
server_key_version: "encryptionkey@0"
I20260926 19:38:09.777040 26831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:09.788055 26831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:09.788909 26831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:09.789227 26831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:09.808132 26831 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:46213
I20260926 19:38:09.808223 26896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:46213 every 8 connection(s)
I20260926 19:38:09.809546 26831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:09.813431 26897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:09.816322 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 26831
I20260926 19:38:09.816507 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:09.817077 7007 external_mini_cluster.cc:1428] Setting key b81ea890aca2453e4cd45aeb9b7533f9
I20260926 19:38:09.825162 26897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97: Bootstrap starting.
I20260926 19:38:09.828173 26897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:09.829299 26897 log.cc:824] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:09.830870 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:09.819055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.6.215.254:46213)
0926 19:38:09.820064 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:09.820074 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:09.820115 (+ 41us) server_negotiation.cc:407] Connection header received
0926 19:38:09.821845 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:09.821868 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:09.822340 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:09.822759 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:09.823342 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:09.824584 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:09.826030 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:09.826739 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:09.826948 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:09.828104 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:09.828115 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:09.828129 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:09.828719 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:09.828868 (+ 149us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:09.828890 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:09.829015 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:09.829252 (+ 237us) server_negotiation.cc:299] Negotiation successful
0926 19:38:09.829593 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":357,"threads_started":1}
I20260926 19:38:09.832671 26897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97: No bootstrap required, opened a new log
I20260926 19:38:09.840169 26897 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1215cfcb6fc4539929e825d52728f97" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46213 } }
I20260926 19:38:09.840543 26897 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:09.840664 26897 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1215cfcb6fc4539929e825d52728f97, State: Initialized, Role: FOLLOWER
I20260926 19:38:09.841306 26897 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1215cfcb6fc4539929e825d52728f97" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46213 } }
I20260926 19:38:09.841539 26897 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:09.841684 26897 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:09.841869 26897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:09.844025 26897 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1215cfcb6fc4539929e825d52728f97" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46213 } }
I20260926 19:38:09.844525 26897 leader_election.cc:302] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1215cfcb6fc4539929e825d52728f97; no voters:
I20260926 19:38:09.845161 26897 leader_election.cc:288] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:09.845316 26902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:09.846168 26902 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [term 1 LEADER]: Becoming Leader. State: Replica: c1215cfcb6fc4539929e825d52728f97, State: Running, Role: LEADER
I20260926 19:38:09.846738 26902 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1215cfcb6fc4539929e825d52728f97" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46213 } }
I20260926 19:38:09.847368 26897 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:09.849399 26903 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1215cfcb6fc4539929e825d52728f97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1215cfcb6fc4539929e825d52728f97" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46213 } } }
I20260926 19:38:09.849746 26903 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:09.850581 26908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:09.850435 26904 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1215cfcb6fc4539929e825d52728f97. Latest consensus state: current_term: 1 leader_uuid: "c1215cfcb6fc4539929e825d52728f97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1215cfcb6fc4539929e825d52728f97" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46213 } } }
I20260926 19:38:09.850745 26904 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:09.857121 26908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:09.867008 26908 catalog_manager.cc:1347] Generated new cluster ID: ae6a1a86f0bf4823aaa4f762e2299b91
I20260926 19:38:09.867165 26908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:09.876878 26908 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:09.878368 26908 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:09.892978 26908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97: Generated new TSK 0
I20260926 19:38:09.893916 26908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:09.901649 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.193:0
--local_ip_for_outbound_sockets=127.6.215.193
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46213
--builtin_ntp_servers=127.6.215.212:38997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 19:38:10.252504 26921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:10.252864 26921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:10.253019 26921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:10.260222 26921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:10.260362 26921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:10.260548 26921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.193
I20260926 19:38:10.269625 26921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.215.193
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.193
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:10.270910 26921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:10.272686 26921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:10.286391 26926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:10.286326 26929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:10.286300 26927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:10.287057 26921 server_base.cc:1034] running on GCE node
I20260926 19:38:10.287688 26921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:10.288584 26921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:10.289788 26921 hybrid_clock.cc:648] HybridClock initialized: now 1790451490289722 us; error 54 us; skew 500 ppm
I20260926 19:38:10.290061 26921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:10.292161 26921 webserver.cc:460] Webserver started at http://127.6.215.193:35617/ using document root <none> and password file <none>
I20260926 19:38:10.292994 26921 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:10.293084 26921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:10.293388 26921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:10.296249 26921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data/instance:
uuid: "fa60a910f6464cf48495de522a9c3dbd"
format_stamp: "Formatted at 2026-09-26 19:38:10 on dist-test-slave-phbh"
server_key: "9e0fc89331f45f7d7a30d4f7abcc2e46"
server_key_iv: "7e2985c2368c4e2210b9867fa7205faf"
server_key_version: "encryptionkey@0"
I20260926 19:38:10.297039 26921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance:
uuid: "fa60a910f6464cf48495de522a9c3dbd"
format_stamp: "Formatted at 2026-09-26 19:38:10 on dist-test-slave-phbh"
server_key: "9e0fc89331f45f7d7a30d4f7abcc2e46"
server_key_iv: "7e2985c2368c4e2210b9867fa7205faf"
server_key_version: "encryptionkey@0"
I20260926 19:38:10.301913 26921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:38:10.305497 26935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:10.306833 26921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:10.307010 26921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal
uuid: "fa60a910f6464cf48495de522a9c3dbd"
format_stamp: "Formatted at 2026-09-26 19:38:10 on dist-test-slave-phbh"
server_key: "9e0fc89331f45f7d7a30d4f7abcc2e46"
server_key_iv: "7e2985c2368c4e2210b9867fa7205faf"
server_key_version: "encryptionkey@0"
I20260926 19:38:10.307243 26921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:10.318890 26921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:10.319677 26921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:10.319993 26921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:10.321031 26921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:10.322485 26921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:10.322613 26921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:10.322749 26921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:10.322839 26921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:10.356287 26921 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.193:44991
I20260926 19:38:10.356338 27047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.193:44991 every 8 connection(s)
I20260926 19:38:10.357857 26921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/data/info.pb
I20260926 19:38:10.366362 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 26921
I20260926 19:38:10.366529 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal/instance
I20260926 19:38:10.367003 7007 external_mini_cluster.cc:1428] Setting key b425e2b91bde7557501afedd81e6046c
I20260926 19:38:10.369603 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.194:0
--local_ip_for_outbound_sockets=127.6.215.194
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46213
--builtin_ntp_servers=127.6.215.212:38997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 19:38:10.370692 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:10.360414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.193:48743 (local address 127.6.215.254:46213)
0926 19:38:10.360943 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:10.360949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:10.362075 (+ 1126us) server_negotiation.cc:407] Connection header received
0926 19:38:10.363792 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:10.363796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:10.363862 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:10.363963 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:10.365029 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.365737 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.367455 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.368017 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.368153 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:10.369635 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:10.369646 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:10.369650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:10.369668 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:10.369707 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:10.369718 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:10.369820 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:10.370130 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 19:38:10.370244 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":235,"threads_started":1}
I20260926 19:38:10.371410 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:10.360992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46213 (local address 127.6.215.193:48743)
0926 19:38:10.361920 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:10.361963 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:10.363535 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:10.364171 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:10.364179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:10.364261 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:10.364859 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:10.364887 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.365873 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.365877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:10.367315 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:10.367327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.368249 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.368257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:10.368510 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:10.369316 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:38:10.369318 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:10.369358 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:38:10.369432 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:10.369872 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:10.369883 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:10.369891 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 19:38:10.370042 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 19:38:10.370339 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":262,"threads_started":1}
I20260926 19:38:10.372900 27048 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46213
I20260926 19:38:10.373208 27048 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:10.373915 27048 heartbeater.cc:511] Master 127.6.215.254:46213 requested a full tablet report, sending...
I20260926 19:38:10.375914 26862 ts_manager.cc:194] Registered new tserver with Master: fa60a910f6464cf48495de522a9c3dbd (127.6.215.193:44991)
I20260926 19:38:10.377975 26862 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.193:48743
W20260926 19:38:10.701220 27052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:10.701514 27052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:10.701654 27052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:10.709429 27052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:10.709544 27052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:10.709717 27052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.194
I20260926 19:38:10.719072 27052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.215.194
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.194
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:10.720441 27052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:10.722349 27052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:10.736011 27057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:10.736167 27058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:10.736166 27060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:10.737327 27052 server_base.cc:1034] running on GCE node
I20260926 19:38:10.738008 27052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:10.739055 27052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:10.740290 27052 hybrid_clock.cc:648] HybridClock initialized: now 1790451490740225 us; error 61 us; skew 500 ppm
I20260926 19:38:10.740638 27052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:10.743160 27052 webserver.cc:460] Webserver started at http://127.6.215.194:38363/ using document root <none> and password file <none>
I20260926 19:38:10.743903 27052 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:10.744041 27052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:10.744369 27052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:10.747532 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data/instance:
uuid: "69df7f8b6f514621a4bd1db41b71861c"
format_stamp: "Formatted at 2026-09-26 19:38:10 on dist-test-slave-phbh"
server_key: "2a617fb65aad155a080916ade1a63491"
server_key_iv: "ee3bbf21aaa7c17257c2bf03cc53a649"
server_key_version: "encryptionkey@0"
I20260926 19:38:10.748430 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance:
uuid: "69df7f8b6f514621a4bd1db41b71861c"
format_stamp: "Formatted at 2026-09-26 19:38:10 on dist-test-slave-phbh"
server_key: "2a617fb65aad155a080916ade1a63491"
server_key_iv: "ee3bbf21aaa7c17257c2bf03cc53a649"
server_key_version: "encryptionkey@0"
I20260926 19:38:10.753963 27052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 19:38:10.757822 27066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:10.759328 27052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:38:10.759516 27052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal
uuid: "69df7f8b6f514621a4bd1db41b71861c"
format_stamp: "Formatted at 2026-09-26 19:38:10 on dist-test-slave-phbh"
server_key: "2a617fb65aad155a080916ade1a63491"
server_key_iv: "ee3bbf21aaa7c17257c2bf03cc53a649"
server_key_version: "encryptionkey@0"
I20260926 19:38:10.759785 27052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:10.780606 27052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:10.781625 27052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:10.781956 27052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:10.783022 27052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:10.784576 27052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:10.784711 27052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:10.784866 27052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:10.784960 27052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:10.819723 27052 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.194:33997
I20260926 19:38:10.819777 27178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.194:33997 every 8 connection(s)
I20260926 19:38:10.821120 27052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/data/info.pb
I20260926 19:38:10.826330 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27052
I20260926 19:38:10.826561 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal/instance
I20260926 19:38:10.826983 7007 external_mini_cluster.cc:1428] Setting key 004b559c70873f7022233c87cb8c1ebb
I20260926 19:38:10.830185 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.215.195:0
--local_ip_for_outbound_sockets=127.6.215.195
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46213
--builtin_ntp_servers=127.6.215.212:38997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 19:38:10.835603 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:10.823850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:45991 (local address 127.6.215.254:46213)
0926 19:38:10.824056 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:10.824062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:10.825627 (+ 1565us) server_negotiation.cc:407] Connection header received
0926 19:38:10.827608 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:10.827616 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:10.827724 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:10.827838 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:10.829336 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.830471 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.832167 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.832882 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.833042 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:10.834632 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:10.834640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:10.834648 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:10.834675 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:10.834722 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:10.834741 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:10.834835 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:10.835062 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 19:38:10.835191 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:38:10.836553 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:10.824363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46213 (local address 127.6.215.194:45991)
0926 19:38:10.825445 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:10.825491 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:10.827329 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:10.828123 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:10.828136 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:10.828258 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:10.829163 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:10.829185 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.830657 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.830661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:10.831991 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:10.832015 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:10.833267 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:10.833283 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:10.833509 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:10.834288 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:38:10.834291 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:10.834333 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:38:10.834424 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:10.834839 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:10.834846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:10.834852 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 19:38:10.835015 (+ 163us) client_negotiation.cc:240] Negotiation successful
0926 19:38:10.835398 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":376,"threads_started":1}
I20260926 19:38:10.838274 27179 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46213
I20260926 19:38:10.838601 27179 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:10.839383 27179 heartbeater.cc:511] Master 127.6.215.254:46213 requested a full tablet report, sending...
I20260926 19:38:10.840984 26862 ts_manager.cc:194] Registered new tserver with Master: 69df7f8b6f514621a4bd1db41b71861c (127.6.215.194:33997)
I20260926 19:38:10.841869 26862 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.194:45991
W20260926 19:38:11.163949 27182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:11.164274 27182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:11.164431 27182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:11.171769 27182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:11.171866 27182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:11.171972 27182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.215.195
I20260926 19:38:11.181201 27182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:38997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.215.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.215.195
--webserver_port=0
--tserver_master_addrs=127.6.215.254:46213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.215.195
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:11.182502 27182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:11.184172 27182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:11.197193 27187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:11.197345 27188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:11.197376 27190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:11.198452 27182 server_base.cc:1034] running on GCE node
I20260926 19:38:11.199007 27182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:11.199901 27182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:11.201133 27182 hybrid_clock.cc:648] HybridClock initialized: now 1790451491201088 us; error 53 us; skew 500 ppm
I20260926 19:38:11.201443 27182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 19:38:11.203610 27182 webserver.cc:460] Webserver started at http://127.6.215.195:35759/ using document root <none> and password file <none>
I20260926 19:38:11.204284 27182 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:11.204416 27182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:11.204733 27182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:11.207509 27182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data/instance:
uuid: "21b3c9ac371e43fbaea6da470eb8e690"
format_stamp: "Formatted at 2026-09-26 19:38:11 on dist-test-slave-phbh"
server_key: "0eaef846a5fc33af9a7a34c694f5bb22"
server_key_iv: "ccbab0b03cd3c8eb4c3447f40315a8c6"
server_key_version: "encryptionkey@0"
I20260926 19:38:11.208360 27182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance:
uuid: "21b3c9ac371e43fbaea6da470eb8e690"
format_stamp: "Formatted at 2026-09-26 19:38:11 on dist-test-slave-phbh"
server_key: "0eaef846a5fc33af9a7a34c694f5bb22"
server_key_iv: "ccbab0b03cd3c8eb4c3447f40315a8c6"
server_key_version: "encryptionkey@0"
I20260926 19:38:11.213351 27182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 19:38:11.216521 27196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:11.217814 27182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:11.217981 27182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal
uuid: "21b3c9ac371e43fbaea6da470eb8e690"
format_stamp: "Formatted at 2026-09-26 19:38:11 on dist-test-slave-phbh"
server_key: "0eaef846a5fc33af9a7a34c694f5bb22"
server_key_iv: "ccbab0b03cd3c8eb4c3447f40315a8c6"
server_key_version: "encryptionkey@0"
I20260926 19:38:11.218204 27182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:11.232746 27182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:11.233704 27182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:11.234056 27182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:11.235037 27182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 19:38:11.236523 27182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 19:38:11.236650 27182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:11.236824 27182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 19:38:11.236918 27182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260926 19:38:11.270903 27182 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.195:35163
I20260926 19:38:11.270954 27308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.195:35163 every 8 connection(s)
I20260926 19:38:11.272210 27182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/data/info.pb
I20260926 19:38:11.279028 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27182
I20260926 19:38:11.279254 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal/instance
I20260926 19:38:11.279685 7007 external_mini_cluster.cc:1428] Setting key 2484d26c8fd61985b0501eecbedf9108
I20260926 19:38:11.285423 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.275066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.195:60895 (local address 127.6.215.254:46213)
0926 19:38:11.275265 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.275270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.276704 (+ 1434us) server_negotiation.cc:407] Connection header received
0926 19:38:11.278497 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.278501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.278567 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.278703 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:11.279910 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.280661 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.282659 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.283071 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.283207 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.284530 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:11.284541 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:11.284544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:11.284562 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:11.284592 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:11.284604 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:11.284702 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.284963 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.285099 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 19:38:11.286365 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.275635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:46213 (local address 127.6.215.195:60895)
0926 19:38:11.276524 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:11.276573 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:11.278261 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:11.278882 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:11.278890 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:11.278971 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:11.279675 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.279692 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.280851 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.280864 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.282507 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.282518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.283286 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.283293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.283515 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.284208 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 19:38:11.284211 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:11.284249 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 19:38:11.284320 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:11.284699 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:11.284711 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:11.284721 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 19:38:11.284953 (+ 232us) client_negotiation.cc:240] Negotiation successful
0926 19:38:11.285347 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":275,"threads_started":1}
I20260926 19:38:11.287684 27309 heartbeater.cc:347] Connected to a master server at 127.6.215.254:46213
I20260926 19:38:11.287997 27309 heartbeater.cc:464] Registering TS with master...
I20260926 19:38:11.288734 27309 heartbeater.cc:511] Master 127.6.215.254:46213 requested a full tablet report, sending...
I20260926 19:38:11.290220 26862 ts_manager.cc:194] Registered new tserver with Master: 21b3c9ac371e43fbaea6da470eb8e690 (127.6.215.195:35163)
I20260926 19:38:11.290957 26862 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.215.195:60895
I20260926 19:38:11.294749 7007 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 19:38:11.304152 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.298854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.6.215.254:46213)
0926 19:38:11.299017 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.299026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.299419 (+ 393us) server_negotiation.cc:407] Connection header received
0926 19:38:11.299543 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.299547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.299615 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.299700 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:11.300152 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.300897 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.302189 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.302585 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.302686 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.303596 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:11.303601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:11.303604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:11.303620 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:11.303661 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:11.303673 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:11.303742 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.303858 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.303926 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 19:38:11.307644 26862 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 19:38:11.314634 26862 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 19:38:11.346251 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.326987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.6.215.194:33997)
0926 19:38:11.337820 (+ 10833us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.337833 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.337857 (+ 24us) server_negotiation.cc:407] Connection header received
0926 19:38:11.337964 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.337974 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.338277 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.339276 (+ 999us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:11.339420 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.340809 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.343923 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.345068 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.345219 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.345499 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.345748 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.345917 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10678,"thread_start_us":10575,"threads_started":1}
I20260926 19:38:11.346299 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.326650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.194:33997 (local address 127.0.0.1:48452)
0926 19:38:11.331800 (+ 5150us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:11.331862 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:11.332072 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:11.338536 (+ 6464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:11.338546 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:11.338579 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:11.339064 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.339079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.340943 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.340948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.343659 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.343674 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.345282 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.345288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.345491 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.345546 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 19:38:11.345704 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 19:38:11.345894 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4815,"mutex_wait_us":99,"thread_start_us":13496,"threads_started":1}
I20260926 19:38:11.346980 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.326927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:44991 (local address 127.0.0.1:50600)
0926 19:38:11.337574 (+ 10647us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:11.337599 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:11.337872 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:11.339255 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:11.339262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:11.339296 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:11.339799 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.339831 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.341697 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.341702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.344023 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.344037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.346194 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.346202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.346386 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.346423 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 19:38:11.346526 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 19:38:11.346650 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10461,"mutex_wait_us":37,"thread_start_us":10086,"threads_started":1}
I20260926 19:38:11.347811 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.327558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50600 (local address 127.6.215.193:44991)
0926 19:38:11.328270 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.328283 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.337733 (+ 9450us) server_negotiation.cc:407] Connection header received
0926 19:38:11.338351 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.338364 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.338657 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.338866 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:11.340018 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.341551 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.344197 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.345708 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.345837 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.346058 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.347246 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.347411 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":337,"threads_started":1}
I20260926 19:38:11.349346 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.330931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:35163 (local address 127.0.0.1:33250)
0926 19:38:11.340402 (+ 9471us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:11.340416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:11.340551 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:11.341176 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:11.341181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:11.341207 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:11.341599 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.341619 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.342915 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.342921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.345296 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.345311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.347641 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.347649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.347842 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.347935 (+ 93us) client_negotiation.cc:769] Sending connection context
0926 19:38:11.348861 (+ 926us) client_negotiation.cc:240] Negotiation successful
0926 19:38:11.349021 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9346,"mutex_wait_us":2,"thread_start_us":354,"threads_started":1}
I20260926 19:38:11.349443 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.331067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33250 (local address 127.6.215.195:35163)
0926 19:38:11.333424 (+ 2357us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.333437 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.340532 (+ 7095us) server_negotiation.cc:407] Connection header received
0926 19:38:11.340691 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.340697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.340946 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.341162 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:11.341788 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.342772 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.345520 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.346933 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.347061 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.347471 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.348916 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.349088 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2125,"thread_start_us":1939,"threads_started":1}
I20260926 19:38:11.350556 27114 tablet_service.cc:1467] Processing CreateTablet for tablet 4dce2506f47c47a683f5ad1aca4b5638 (DEFAULT_TABLE table=test-table [id=1b57969b538644479a296d47b438cabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:11.351374 26983 tablet_service.cc:1467] Processing CreateTablet for tablet 4dce2506f47c47a683f5ad1aca4b5638 (DEFAULT_TABLE table=test-table [id=1b57969b538644479a296d47b438cabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:11.352445 27114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dce2506f47c47a683f5ad1aca4b5638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:11.352469 26983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dce2506f47c47a683f5ad1aca4b5638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:11.353191 27244 tablet_service.cc:1467] Processing CreateTablet for tablet 4dce2506f47c47a683f5ad1aca4b5638 (DEFAULT_TABLE table=test-table [id=1b57969b538644479a296d47b438cabf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 19:38:11.354525 27244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dce2506f47c47a683f5ad1aca4b5638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:11.369398 27328 tablet_bootstrap.cc:492] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Bootstrap starting.
I20260926 19:38:11.373252 27327 tablet_bootstrap.cc:492] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Bootstrap starting.
I20260926 19:38:11.373458 27328 tablet_bootstrap.cc:654] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:11.373524 27329 tablet_bootstrap.cc:492] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Bootstrap starting.
I20260926 19:38:11.374873 27328 log.cc:824] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:11.377034 27329 tablet_bootstrap.cc:654] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:11.377672 27327 tablet_bootstrap.cc:654] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:11.378299 27328 tablet_bootstrap.cc:492] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: No bootstrap required, opened a new log
I20260926 19:38:11.378336 27329 log.cc:824] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:11.378619 27328 ts_tablet_manager.cc:1397] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 19:38:11.378674 27327 log.cc:824] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:11.381057 27048 heartbeater.cc:503] Master 127.6.215.254:46213 was elected leader, sending a full tablet report...
I20260926 19:38:11.381696 27329 tablet_bootstrap.cc:492] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: No bootstrap required, opened a new log
I20260926 19:38:11.381887 27327 tablet_bootstrap.cc:492] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: No bootstrap required, opened a new log
I20260926 19:38:11.381969 27329 ts_tablet_manager.cc:1397] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260926 19:38:11.382148 27327 ts_tablet_manager.cc:1397] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 19:38:11.390165 27328 raft_consensus.cc:348] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.390579 27328 raft_consensus.cc:374] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:11.390726 27328 raft_consensus.cc:729] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa60a910f6464cf48495de522a9c3dbd, State: Initialized, Role: FOLLOWER
I20260926 19:38:11.391556 27328 consensus_queue.cc:260] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.392823 27327 raft_consensus.cc:348] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.392829 27329 raft_consensus.cc:348] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.393440 27329 raft_consensus.cc:374] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:11.393467 27327 raft_consensus.cc:374] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:11.393582 27327 raft_consensus.cc:729] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69df7f8b6f514621a4bd1db41b71861c, State: Initialized, Role: FOLLOWER
I20260926 19:38:11.393611 27329 raft_consensus.cc:729] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21b3c9ac371e43fbaea6da470eb8e690, State: Initialized, Role: FOLLOWER
I20260926 19:38:11.394228 27329 consensus_queue.cc:260] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.394227 27327 consensus_queue.cc:260] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.394827 27328 ts_tablet_manager.cc:1428] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260926 19:38:11.395812 27179 heartbeater.cc:503] Master 127.6.215.254:46213 was elected leader, sending a full tablet report...
I20260926 19:38:11.396656 27327 ts_tablet_manager.cc:1428] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260926 19:38:11.396940 27309 heartbeater.cc:503] Master 127.6.215.254:46213 was elected leader, sending a full tablet report...
I20260926 19:38:11.397723 27329 ts_tablet_manager.cc:1428] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
W20260926 19:38:11.524067 27310 tablet.cc:2369] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:38:11.573521 27180 tablet.cc:2369] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:38:11.610453 27049 tablet.cc:2369] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:11.680425 27335 raft_consensus.cc:482] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 19:38:11.680822 27335 raft_consensus.cc:504] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.682736 27335 leader_election.cc:288] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa60a910f6464cf48495de522a9c3dbd (127.6.215.193:44991), 21b3c9ac371e43fbaea6da470eb8e690 (127.6.215.195:35163)
I20260926 19:38:11.692385 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.683410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:40083 (local address 127.6.215.193:44991)
0926 19:38:11.683645 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.683651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.684149 (+ 498us) server_negotiation.cc:407] Connection header received
0926 19:38:11.684478 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.684484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.684582 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.684698 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:11.685656 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.686758 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.689953 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.691214 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.691393 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.691466 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.691960 (+ 494us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.692108 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 19:38:11.692385 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.683915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.215.194:38693 (local address 127.6.215.195:35163)
0926 19:38:11.684163 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.684168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.684432 (+ 264us) server_negotiation.cc:407] Connection header received
0926 19:38:11.684577 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.684582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.684666 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.684755 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 19:38:11.685656 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.686840 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.689953 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.691269 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.691416 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.691475 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.691960 (+ 485us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.692151 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260926 19:38:11.692518 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.683608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.195:35163 (local address 127.6.215.194:38693)
0926 19:38:11.684323 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:11.684347 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:11.684451 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:11.684805 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:11.684809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:11.684826 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:11.685423 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.685442 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.686982 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.686989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.689712 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.689728 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.691495 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.691502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.691708 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.691818 (+ 110us) client_negotiation.cc:769] Sending connection context
0926 19:38:11.691910 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 19:38:11.692092 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":280,"thread_start_us":417,"threads_started":1}
I20260926 19:38:11.692796 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.683107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.193:44991 (local address 127.6.215.194:40083)
0926 19:38:11.684011 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:11.684035 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:11.684281 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:11.684751 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:11.684757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:11.684808 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 19:38:11.685487 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.685501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.686954 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.686961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.689712 (+ 2751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:11.689728 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.691501 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.691508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:11.691705 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.691811 (+ 106us) client_negotiation.cc:769] Sending connection context
0926 19:38:11.691910 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 19:38:11.692092 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":288,"thread_start_us":533,"threads_started":1}
I20260926 19:38:11.693723 27264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dce2506f47c47a683f5ad1aca4b5638" candidate_uuid: "69df7f8b6f514621a4bd1db41b71861c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21b3c9ac371e43fbaea6da470eb8e690" is_pre_election: true
I20260926 19:38:11.693806 27003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dce2506f47c47a683f5ad1aca4b5638" candidate_uuid: "69df7f8b6f514621a4bd1db41b71861c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa60a910f6464cf48495de522a9c3dbd" is_pre_election: true
I20260926 19:38:11.694361 27003 raft_consensus.cc:2454] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69df7f8b6f514621a4bd1db41b71861c in term 0.
I20260926 19:38:11.694360 27264 raft_consensus.cc:2454] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69df7f8b6f514621a4bd1db41b71861c in term 0.
I20260926 19:38:11.695144 27067 leader_election.cc:302] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21b3c9ac371e43fbaea6da470eb8e690, 69df7f8b6f514621a4bd1db41b71861c, fa60a910f6464cf48495de522a9c3dbd; no voters:
I20260926 19:38:11.695607 27335 raft_consensus.cc:2781] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 19:38:11.695832 27335 raft_consensus.cc:482] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 19:38:11.695960 27335 raft_consensus.cc:3037] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:11.698282 27335 raft_consensus.cc:504] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.698890 27335 leader_election.cc:288] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [CANDIDATE]: Term 1 election: Requested vote from peers fa60a910f6464cf48495de522a9c3dbd (127.6.215.193:44991), 21b3c9ac371e43fbaea6da470eb8e690 (127.6.215.195:35163)
I20260926 19:38:11.699397 27003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dce2506f47c47a683f5ad1aca4b5638" candidate_uuid: "69df7f8b6f514621a4bd1db41b71861c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa60a910f6464cf48495de522a9c3dbd"
I20260926 19:38:11.699383 27264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dce2506f47c47a683f5ad1aca4b5638" candidate_uuid: "69df7f8b6f514621a4bd1db41b71861c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21b3c9ac371e43fbaea6da470eb8e690"
I20260926 19:38:11.699633 27003 raft_consensus.cc:3037] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:11.699635 27264 raft_consensus.cc:3037] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:11.702144 27003 raft_consensus.cc:2454] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69df7f8b6f514621a4bd1db41b71861c in term 1.
I20260926 19:38:11.702147 27264 raft_consensus.cc:2454] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69df7f8b6f514621a4bd1db41b71861c in term 1.
I20260926 19:38:11.702548 27067 leader_election.cc:302] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69df7f8b6f514621a4bd1db41b71861c, fa60a910f6464cf48495de522a9c3dbd; no voters:
I20260926 19:38:11.702867 27335 raft_consensus.cc:2781] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:11.703260 27335 raft_consensus.cc:686] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 1 LEADER]: Becoming Leader. State: Replica: 69df7f8b6f514621a4bd1db41b71861c, State: Running, Role: LEADER
I20260926 19:38:11.703843 27335 consensus_queue.cc:237] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } }
I20260926 19:38:11.708290 26860 catalog_manager.cc:5495] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c reported cstate change: term changed from 0 to 1, leader changed from <none> to 69df7f8b6f514621a4bd1db41b71861c (127.6.215.194). New cstate: current_term: 1 leader_uuid: "69df7f8b6f514621a4bd1db41b71861c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69df7f8b6f514621a4bd1db41b71861c" member_type: VOTER last_known_addr { host: "127.6.215.194" port: 33997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 } health_report { overall_health: UNKNOWN } } }
I20260926 19:38:11.801906 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.796346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.6.215.194:33997)
0926 19:38:11.796728 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.796735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.796755 (+ 20us) server_negotiation.cc:407] Connection header received
0926 19:38:11.796981 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.797006 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.797151 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.797309 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:11.797735 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.798405 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.799748 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.800160 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.800256 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.800725 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:11.801308 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:11.801417 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.801595 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.801677 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260926 19:38:11.810737 27264 raft_consensus.cc:1261] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 1 FOLLOWER]: Refusing update from remote peer 69df7f8b6f514621a4bd1db41b71861c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:11.810936 27003 raft_consensus.cc:1261] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 1 FOLLOWER]: Refusing update from remote peer 69df7f8b6f514621a4bd1db41b71861c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:11.811892 27338 consensus_queue.cc:1035] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [LEADER]: Connected to new peer: Peer: permanent_uuid: "21b3c9ac371e43fbaea6da470eb8e690" member_type: VOTER last_known_addr { host: "127.6.215.195" port: 35163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:11.812258 27335 consensus_queue.cc:1035] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa60a910f6464cf48495de522a9c3dbd" member_type: VOTER last_known_addr { host: "127.6.215.193" port: 44991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:11.831856 27341 mvcc.cc:205] Tried to move back new op lower bound from 7333689310445420544 to 7333689310022893568. Current Snapshot: MvccSnapshot[applied={T|T < 7333689310445420544}]
I20260926 19:38:11.833262 27342 mvcc.cc:205] Tried to move back new op lower bound from 7333689310445420544 to 7333689310022893568. Current Snapshot: MvccSnapshot[applied={T|T < 7333689310445420544}]
I20260926 19:38:11.848207 27343 mvcc.cc:205] Tried to move back new op lower bound from 7333689310445420544 to 7333689310022893568. Current Snapshot: MvccSnapshot[applied={T|T < 7333689310445420544}]
I20260926 19:38:11.860342 26859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:11.860585 26859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 19:38:11.863963 26859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1215cfcb6fc4539929e825d52728f97: Sending DeleteTablet for 3 replicas of tablet 4dce2506f47c47a683f5ad1aca4b5638
I20260926 19:38:11.865182 26983 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dce2506f47c47a683f5ad1aca4b5638 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:11 UTC) from {username='slave'} at 127.0.0.1:50600
I20260926 19:38:11.865387 27244 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dce2506f47c47a683f5ad1aca4b5638 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:11 UTC) from {username='slave'} at 127.0.0.1:33250
I20260926 19:38:11.865971 27114 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dce2506f47c47a683f5ad1aca4b5638 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 19:38:11 UTC) from {username='slave'} at 127.0.0.1:48452
I20260926 19:38:11.867828 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 26921
I20260926 19:38:11.880868 27351 tablet_replica.cc:330] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: stopping tablet replica
I20260926 19:38:11.881567 27351 raft_consensus.cc:2229] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:11.882378 27351 raft_consensus.cc:2258] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:11.885437 27351 ts_tablet_manager.cc:1905] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:11.895506 27351 ts_tablet_manager.cc:1918] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:11.895813 27351 log.cc:1196] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-0/wal/wals/4dce2506f47c47a683f5ad1aca4b5638
I20260926 19:38:11.896629 27351 ts_tablet_manager.cc:1939] T 4dce2506f47c47a683f5ad1aca4b5638 P fa60a910f6464cf48495de522a9c3dbd: Deleting consensus metadata
I20260926 19:38:11.899272 26846 catalog_manager.cc:4841] TS fa60a910f6464cf48495de522a9c3dbd (127.6.215.193:44991): tablet 4dce2506f47c47a683f5ad1aca4b5638 (table test-table [id=1b57969b538644479a296d47b438cabf]) successfully deleted
I20260926 19:38:11.895623 27353 tablet_replica.cc:330] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: stopping tablet replica
I20260926 19:38:11.900158 27353 raft_consensus.cc:2229] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 1 LEADER]: Raft consensus shutting down.
I20260926 19:38:11.900961 27353 raft_consensus.cc:2258] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:11.902594 27352 tablet_replica.cc:330] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: stopping tablet replica
I20260926 19:38:11.903049 27352 raft_consensus.cc:2229] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 19:38:11.903579 27352 raft_consensus.cc:2258] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 19:38:11.904381 27353 ts_tablet_manager.cc:1905] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:11.905247 27352 ts_tablet_manager.cc:1905] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 19:38:11.910053 27352 ts_tablet_manager.cc:1918] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:11.910359 27352 log.cc:1196] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-2/wal/wals/4dce2506f47c47a683f5ad1aca4b5638
I20260926 19:38:11.910892 27352 ts_tablet_manager.cc:1939] T 4dce2506f47c47a683f5ad1aca4b5638 P 21b3c9ac371e43fbaea6da470eb8e690: Deleting consensus metadata
I20260926 19:38:11.912628 26848 catalog_manager.cc:4841] TS 21b3c9ac371e43fbaea6da470eb8e690 (127.6.215.195:35163): tablet 4dce2506f47c47a683f5ad1aca4b5638 (table test-table [id=1b57969b538644479a296d47b438cabf]) successfully deleted
I20260926 19:38:11.912739 27353 ts_tablet_manager.cc:1918] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 19:38:11.913005 27353 log.cc:1196] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Deleting WAL directory at /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790451176367082-7007-0/minicluster-data/ts-1/wal/wals/4dce2506f47c47a683f5ad1aca4b5638
I20260926 19:38:11.913609 27353 ts_tablet_manager.cc:1939] T 4dce2506f47c47a683f5ad1aca4b5638 P 69df7f8b6f514621a4bd1db41b71861c: Deleting consensus metadata
I20260926 19:38:11.915369 26849 catalog_manager.cc:4841] TS 69df7f8b6f514621a4bd1db41b71861c (127.6.215.194:33997): tablet 4dce2506f47c47a683f5ad1aca4b5638 (table test-table [id=1b57969b538644479a296d47b438cabf]) successfully deleted
I20260926 19:38:11.920755 27324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:11.869439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.6.215.193:44991)
0926 19:38:11.869634 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:11.869641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:11.893667 (+ 24026us) server_negotiation.cc:407] Connection header received
0926 19:38:11.894217 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:11.894224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:11.894364 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:11.894457 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:11.895865 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.897185 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.902054 (+ 4869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:11.902758 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:11.902912 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:11.918863 (+ 15951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:11.918881 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:11.918907 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:11.919808 (+ 901us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:11.919899 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:11.919938 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:11.920117 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:11.920221 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 19:38:11.920360 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:38:11.921607 27043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:12.013456 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 26921
I20260926 19:38:12.040001 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27052
I20260926 19:38:12.049278 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:12.040754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.6.215.194:33997)
0926 19:38:12.040949 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:12.040954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:12.041082 (+ 128us) server_negotiation.cc:407] Connection header received
0926 19:38:12.041281 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:12.041285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:12.041375 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:12.041455 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:12.041959 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:12.042751 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:12.044855 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:12.045471 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:12.045568 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:12.046823 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:12.046832 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:12.046846 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:12.047691 (+ 845us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:12.047766 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:12.047790 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:12.047897 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:12.048302 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 19:38:12.048479 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 19:38:12.050442 27174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:12.161576 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27052
I20260926 19:38:12.180569 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27182
I20260926 19:38:12.187721 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:12.181367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.6.215.195:35163)
0926 19:38:12.181562 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:12.181570 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:12.181681 (+ 111us) server_negotiation.cc:407] Connection header received
0926 19:38:12.181889 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:12.181895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:12.181999 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:12.182111 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:12.182528 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:12.183257 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:12.184578 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:12.185062 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:12.185199 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:12.186360 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:12.186370 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:12.186383 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:12.186941 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:12.186991 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:12.187010 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:12.187135 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:12.187273 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 19:38:12.187378 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:38:12.188355 27304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:12.275987 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27182
I20260926 19:38:12.292475 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 26831
I20260926 19:38:12.293610 26892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:12.402930 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 26831
2026-09-26T19:38:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3093 ms)
[----------] 6 tests from EncryptionPolicyTest (21805 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:12 dist-test-slave-phbh krb5kdc[27361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:12 dist-test-slave-phbh krb5kdc[27361](info): set up 2 sockets
Sep 26 19:38:12 dist-test-slave-phbh krb5kdc[27361](info): commencing operation
krb5kdc: starting...
W20260926 19:38:13.581442 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:13 dist-test-slave-phbh krb5kdc[27361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:13.832863 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:14.164428 27374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:14.164719 27374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:14.164956 27374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:14.172245 27374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:14.172364 27374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:14.172470 27374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:14.172578 27374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:14.172667 27374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:14.182837 27374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:14.184239 27374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:14.185634 27374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:14.197925 27380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:14.197954 27382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:14.198410 27379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:14.199132 27374 server_base.cc:1034] running on GCE node
I20260926 19:38:14.199913 27374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:14.201417 27374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:14.202657 27374 hybrid_clock.cc:648] HybridClock initialized: now 1790451494202610 us; error 37 us; skew 500 ppm
I20260926 19:38:14.203012 27374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:14 dist-test-slave-phbh krb5kdc[27361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451494, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:14.206149 27374 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:14.207671 27374 webserver.cc:460] Webserver started at http://127.0.0.1:39449/ using document root <none> and password file <none>
I20260926 19:38:14.208377 27374 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:14.208523 27374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:14.208891 27374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:14.211895 27374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd"
format_stamp: "Formatted at 2026-09-26 19:38:14 on dist-test-slave-phbh"
server_key: "3fda98edc88ef6998cbd4353833dd494"
server_key_iv: "85cacf4aa6c9be7dd140d197d59f52e2"
server_key_version: "encryptionkey@0"
I20260926 19:38:14.212679 27374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd"
format_stamp: "Formatted at 2026-09-26 19:38:14 on dist-test-slave-phbh"
server_key: "3fda98edc88ef6998cbd4353833dd494"
server_key_iv: "85cacf4aa6c9be7dd140d197d59f52e2"
server_key_version: "encryptionkey@0"
I20260926 19:38:14.218103 27374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:38:14.221608 27389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:14.222982 27374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:14.223178 27374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd"
format_stamp: "Formatted at 2026-09-26 19:38:14 on dist-test-slave-phbh"
server_key: "3fda98edc88ef6998cbd4353833dd494"
server_key_iv: "85cacf4aa6c9be7dd140d197d59f52e2"
server_key_version: "encryptionkey@0"
I20260926 19:38:14.223390 27374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:14.240911 27374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:14.245638 27374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:14.245887 27374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:14.264535 27374 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46853
I20260926 19:38:14.264569 27440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46853 every 8 connection(s)
I20260926 19:38:14.265965 27374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:14.267247 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27374
I20260926 19:38:14.267467 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:14.268132 7007 external_mini_cluster.cc:1428] Setting key 15f0b2c7e2a4dcb3a6976979a917febe
I20260926 19:38:14.270723 27441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 19:38:14 dist-test-slave-phbh krb5kdc[27361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 19:38:14.283525 27441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd: Bootstrap starting.
I20260926 19:38:14.287794 27441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:14.289134 27441 log.cc:824] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:14.292690 27441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd: No bootstrap required, opened a new log
I20260926 19:38:14.293725 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:14.270044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34913 (local address 127.0.0.1:46853)
0926 19:38:14.270832 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:14.270846 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:14.270893 (+ 47us) server_negotiation.cc:407] Connection header received
0926 19:38:14.273273 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:14.273314 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:14.273835 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:14.274285 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:14.275701 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:14.277813 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:14.279124 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:14.280148 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:14.280326 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:14.284459 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:14.284470 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:14.284487 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:14.284547 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:14.288563 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:14.289493 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:14.289501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:14.289509 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:14.289738 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:14.290368 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:14.290372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:14.290374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:14.291155 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:14.291442 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:14.291927 (+ 485us) server_negotiation.cc:299] Negotiation successful
0926 19:38:14.292280 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":327,"threads_started":1}
I20260926 19:38:14.297897 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:14.302009 27441 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46853 } }
I20260926 19:38:14.302376 27441 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:14.302495 27441 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4eabe5d6a9f4a8fad3ebf9ad068dbcd, State: Initialized, Role: FOLLOWER
I20260926 19:38:14.303211 27441 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46853 } }
I20260926 19:38:14.303514 27441 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:14.303692 27441 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:14.303879 27441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:14.306666 27441 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46853 } }
I20260926 19:38:14.307196 27441 leader_election.cc:302] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4eabe5d6a9f4a8fad3ebf9ad068dbcd; no voters:
I20260926 19:38:14.307879 27441 leader_election.cc:288] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:14.307981 27451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:14.310809 27451 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [term 1 LEADER]: Becoming Leader. State: Replica: f4eabe5d6a9f4a8fad3ebf9ad068dbcd, State: Running, Role: LEADER
I20260926 19:38:14.311292 27441 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:14.311546 27451 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46853 } }
I20260926 19:38:14.316483 27452 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46853 } } }
I20260926 19:38:14.316920 27452 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:14.317487 27453 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4eabe5d6a9f4a8fad3ebf9ad068dbcd. Latest consensus state: current_term: 1 leader_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eabe5d6a9f4a8fad3ebf9ad068dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46853 } } }
I20260926 19:38:14.317889 27453 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:14.321329 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:14.303292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37463 (local address 127.0.0.1:46853)
0926 19:38:14.303590 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:14.303597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:14.303923 (+ 326us) server_negotiation.cc:407] Connection header received
0926 19:38:14.304110 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:14.304116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:14.304203 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:14.304315 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:14.305678 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:14.306808 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:14.308177 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:14.308893 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:14.309004 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:14.311796 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:14.311803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:14.311807 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:14.311844 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:14.315891 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:14.317119 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:14.317125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:14.317129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:14.317351 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:14.318092 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:14.318100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:14.318106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:14.318611 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:14.318975 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:14.320508 (+ 1533us) server_negotiation.cc:299] Negotiation successful
0926 19:38:14.320801 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:38:14.325125 27457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:14.331009 27457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:14.340617 27457 catalog_manager.cc:1347] Generated new cluster ID: 078e80519aac4d05bfcaf83b59898d1f
I20260926 19:38:14.340754 27457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:14.355119 27457 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:14.356452 27457 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:14.363061 27457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4eabe5d6a9f4a8fad3ebf9ad068dbcd: Generated new TSK 0
I20260926 19:38:14.363940 27457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:14.372575 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27374
I20260926 19:38:14.374804 27436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:14.466717 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27374
W20260926 19:38:14.484040 27359 connection.cc:535] client connection to 127.0.0.1:46853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46853): Connection reset by peer (error 104)
2026-09-26T19:38:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:14 dist-test-slave-phbh krb5kdc[27473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:14 dist-test-slave-phbh krb5kdc[27473](info): set up 2 sockets
Sep 26 19:38:14 dist-test-slave-phbh krb5kdc[27473](info): commencing operation
krb5kdc: starting...
W20260926 19:38:15.613296 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:15 dist-test-slave-phbh krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:15.868315 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:16.198215 27486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:16.198493 27486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:16.198588 27486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:16.205756 27486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:16.205870 27486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:16.205924 27486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:16.205986 27486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:16.206030 27486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:16.215665 27486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:16.216935 27486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:16.218149 27486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:16.232977 27492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:16.233215 27494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:16.233265 27491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:16.234140 27486 server_base.cc:1034] running on GCE node
I20260926 19:38:16.234911 27486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:16.236421 27486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:16.237656 27486 hybrid_clock.cc:648] HybridClock initialized: now 1790451496237593 us; error 52 us; skew 500 ppm
I20260926 19:38:16.238013 27486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:16 dist-test-slave-phbh krb5kdc[27473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451496, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:16.241374 27486 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:16.242857 27486 webserver.cc:460] Webserver started at http://127.0.0.1:43719/ using document root <none> and password file <none>
I20260926 19:38:16.243569 27486 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:16.243732 27486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:16.244038 27486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:16.247262 27486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "f01da8cc8d3e4816a082e4fde916907b"
format_stamp: "Formatted at 2026-09-26 19:38:16 on dist-test-slave-phbh"
server_key: "2c19d5e3f5bcf359204b29ed57a082ba"
server_key_iv: "5b39f2d089a1e466263df5008e2f7457"
server_key_version: "encryptionkey@0"
I20260926 19:38:16.248013 27486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "f01da8cc8d3e4816a082e4fde916907b"
format_stamp: "Formatted at 2026-09-26 19:38:16 on dist-test-slave-phbh"
server_key: "2c19d5e3f5bcf359204b29ed57a082ba"
server_key_iv: "5b39f2d089a1e466263df5008e2f7457"
server_key_version: "encryptionkey@0"
I20260926 19:38:16.253264 27486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 19:38:16.256860 27501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:16.258123 27486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:16.258320 27486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "f01da8cc8d3e4816a082e4fde916907b"
format_stamp: "Formatted at 2026-09-26 19:38:16 on dist-test-slave-phbh"
server_key: "2c19d5e3f5bcf359204b29ed57a082ba"
server_key_iv: "5b39f2d089a1e466263df5008e2f7457"
server_key_version: "encryptionkey@0"
I20260926 19:38:16.258522 27486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:16.282538 27486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:16.286784 27486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:16.287086 27486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:16.305737 27486 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40427
I20260926 19:38:16.305764 27552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40427 every 8 connection(s)
I20260926 19:38:16.307214 27486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:16.310832 27553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:16.317508 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27486
I20260926 19:38:16.317724 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:16.318172 7007 external_mini_cluster.cc:1428] Setting key 0633ffc9df96d9730a6103c77d8aa890
I20260926 19:38:16.321319 27553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b: Bootstrap starting.
I20260926 19:38:16.324604 27553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:16.325778 27553 log.cc:824] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:16.328850 27553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b: No bootstrap required, opened a new log
I20260926 19:38:16.331815 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:16.320038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46379 (local address 127.0.0.1:40427)
0926 19:38:16.320904 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:16.320915 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:16.320959 (+ 44us) server_negotiation.cc:407] Connection header received
0926 19:38:16.322761 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:16.322786 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:16.323175 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:16.323490 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:16.324124 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:16.325559 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:16.327301 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:16.327955 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:16.328088 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:16.329499 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:16.329507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:16.329521 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:16.330074 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:16.330151 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:16.330175 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:16.330366 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:16.330670 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 19:38:16.330835 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":372,"threads_started":1}
I20260926 19:38:16.335734 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:16.337123 27553 raft_consensus.cc:348] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01da8cc8d3e4816a082e4fde916907b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260926 19:38:16.337522 27553 raft_consensus.cc:374] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:16.337626 27553 raft_consensus.cc:729] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01da8cc8d3e4816a082e4fde916907b, State: Initialized, Role: FOLLOWER
I20260926 19:38:16.338399 27553 consensus_queue.cc:260] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01da8cc8d3e4816a082e4fde916907b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260926 19:38:16.338678 27553 raft_consensus.cc:388] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:16.338847 27553 raft_consensus.cc:482] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:16.339035 27553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:16.341944 27553 raft_consensus.cc:504] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01da8cc8d3e4816a082e4fde916907b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260926 19:38:16.342458 27553 leader_election.cc:302] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f01da8cc8d3e4816a082e4fde916907b; no voters:
I20260926 19:38:16.343145 27553 leader_election.cc:288] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:16.343261 27563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:16.345825 27553 sys_catalog.cc:564] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:16.346724 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:16.340193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40691 (local address 127.0.0.1:40427)
0926 19:38:16.340390 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:16.340396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:16.340759 (+ 363us) server_negotiation.cc:407] Connection header received
0926 19:38:16.340989 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:16.340994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:16.341076 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:16.341189 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:16.341654 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:16.342469 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:16.344122 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:16.344595 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:16.344745 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:16.345913 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:16.345921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:16.345943 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:16.345967 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:16.346038 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:16.346068 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:16.346200 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:16.346393 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 19:38:16.346448 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 19:38:16.350310 27563 raft_consensus.cc:686] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [term 1 LEADER]: Becoming Leader. State: Replica: f01da8cc8d3e4816a082e4fde916907b, State: Running, Role: LEADER
I20260926 19:38:16.350993 27563 consensus_queue.cc:237] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01da8cc8d3e4816a082e4fde916907b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260926 19:38:16.354375 27564 sys_catalog.cc:455] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f01da8cc8d3e4816a082e4fde916907b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01da8cc8d3e4816a082e4fde916907b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } } }
I20260926 19:38:16.354792 27564 sys_catalog.cc:458] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:16.356423 27578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:16.356338 27565 sys_catalog.cc:455] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f01da8cc8d3e4816a082e4fde916907b. Latest consensus state: current_term: 1 leader_uuid: "f01da8cc8d3e4816a082e4fde916907b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01da8cc8d3e4816a082e4fde916907b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } } }
I20260926 19:38:16.356630 27565 sys_catalog.cc:458] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:16.360733 27578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:16.369038 27578 catalog_manager.cc:1347] Generated new cluster ID: 86fab3b815f24744bad48e06a42e204c
I20260926 19:38:16.369172 27578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:16.388738 27578 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:16.390157 27578 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:16.396903 27578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f01da8cc8d3e4816a082e4fde916907b: Generated new TSK 0
I20260926 19:38:16.397778 27578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:16.411233 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27486
W20260926 19:38:16.412358 27548 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46379
W20260926 19:38:16.413184 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27486: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:16.413369 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27486
W20260926 19:38:16.430894 27471 connection.cc:535] client connection to 127.0.0.1:40427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40427): Connection reset by peer (error 104)
2026-09-26T19:38:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1946 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:16 dist-test-slave-phbh krb5kdc[27584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:16 dist-test-slave-phbh krb5kdc[27584](info): set up 2 sockets
Sep 26 19:38:16 dist-test-slave-phbh krb5kdc[27584](info): commencing operation
krb5kdc: starting...
W20260926 19:38:17.521193 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:17 dist-test-slave-phbh krb5kdc[27584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:17.770119 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:18.094506 27597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:18.094841 27597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:18.095037 27597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:18.102514 27597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:18.102685 27597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:18.102787 27597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:18.102886 27597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:18.102952 27597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:18.113219 27597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37181
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:18.114532 27597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:18.115828 27597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:18.127923 27605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:18.127876 27603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:18.127997 27602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:18.129216 27597 server_base.cc:1034] running on GCE node
I20260926 19:38:18.130081 27597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:18.131608 27597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:18.132908 27597 hybrid_clock.cc:648] HybridClock initialized: now 1790451498132865 us; error 57 us; skew 500 ppm
I20260926 19:38:18.133311 27597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:18 dist-test-slave-phbh krb5kdc[27584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451498, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:18.149009 27597 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:18.150686 27597 webserver.cc:460] Webserver started at http://127.0.0.1:42097/ using document root <none> and password file <none>
I20260926 19:38:18.151391 27597 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:18.151494 27597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:18.151842 27597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:18.155324 27597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "b70c49504e9a445cb0dd5d5acd1e06b0"
format_stamp: "Formatted at 2026-09-26 19:38:18 on dist-test-slave-phbh"
server_key: "7629796b960a8be06127f4a5e505b807"
server_key_iv: "58451e59d8667efb259047f8638b591a"
server_key_version: "encryptionkey@0"
I20260926 19:38:18.156042 27597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "b70c49504e9a445cb0dd5d5acd1e06b0"
format_stamp: "Formatted at 2026-09-26 19:38:18 on dist-test-slave-phbh"
server_key: "7629796b960a8be06127f4a5e505b807"
server_key_iv: "58451e59d8667efb259047f8638b591a"
server_key_version: "encryptionkey@0"
I20260926 19:38:18.161221 27597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 19:38:18.164887 27612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:18.166234 27597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:18.166412 27597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "b70c49504e9a445cb0dd5d5acd1e06b0"
format_stamp: "Formatted at 2026-09-26 19:38:18 on dist-test-slave-phbh"
server_key: "7629796b960a8be06127f4a5e505b807"
server_key_iv: "58451e59d8667efb259047f8638b591a"
server_key_version: "encryptionkey@0"
I20260926 19:38:18.166604 27597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:18.172503 27597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:18.176635 27597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:18.176961 27597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:18.195303 27597 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37181
I20260926 19:38:18.195382 27663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37181 every 8 connection(s)
I20260926 19:38:18.196786 27597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:18.200343 27664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:18.204854 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27597
I20260926 19:38:18.205032 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:18.205600 7007 external_mini_cluster.cc:1428] Setting key 5c035341bc20a1ca4b0dde8fcf2f922d
I20260926 19:38:18.211594 27664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0: Bootstrap starting.
I20260926 19:38:18.213536 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:18.207369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38243 (local address 127.0.0.1:37181)
0926 19:38:18.208158 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:18.208169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:18.208207 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:38:18.210175 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:18.210198 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:18.210513 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:18.210817 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:18.211959 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:18.211966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:18.211978 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:18.211998 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:18.212056 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:18.212077 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:18.212303 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:18.212402 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 19:38:18.212584 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":364,"threads_started":1}
I20260926 19:38:18.214879 27664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:18.215971 27664 log.cc:824] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:18.217198 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:18.218699 27664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0: No bootstrap required, opened a new log
I20260926 19:38:18.223917 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:18.221548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47189 (local address 127.0.0.1:37181)
0926 19:38:18.221725 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:18.221731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:18.222129 (+ 398us) server_negotiation.cc:407] Connection header received
0926 19:38:18.222251 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:18.222256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:18.222312 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:18.222411 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:18.223281 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:18.223287 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:18.223291 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:18.223310 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:18.223362 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:18.223388 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:18.223505 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:18.223575 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 19:38:18.223633 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 19:38:18.227183 27664 raft_consensus.cc:348] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37181 } }
I20260926 19:38:18.227591 27664 raft_consensus.cc:374] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:18.227721 27664 raft_consensus.cc:729] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b70c49504e9a445cb0dd5d5acd1e06b0, State: Initialized, Role: FOLLOWER
I20260926 19:38:18.228278 27664 consensus_queue.cc:260] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37181 } }
I20260926 19:38:18.228541 27664 raft_consensus.cc:388] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:18.228698 27664 raft_consensus.cc:482] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:18.228907 27664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:18.231153 27664 raft_consensus.cc:504] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37181 } }
I20260926 19:38:18.231654 27664 leader_election.cc:302] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b70c49504e9a445cb0dd5d5acd1e06b0; no voters:
I20260926 19:38:18.232295 27664 leader_election.cc:288] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:18.232452 27674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:18.233227 27674 raft_consensus.cc:686] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [term 1 LEADER]: Becoming Leader. State: Replica: b70c49504e9a445cb0dd5d5acd1e06b0, State: Running, Role: LEADER
I20260926 19:38:18.233897 27674 consensus_queue.cc:237] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37181 } }
I20260926 19:38:18.235021 27664 sys_catalog.cc:564] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:18.237346 27675 sys_catalog.cc:455] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b70c49504e9a445cb0dd5d5acd1e06b0. Latest consensus state: current_term: 1 leader_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37181 } } }
I20260926 19:38:18.237835 27675 sys_catalog.cc:458] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:18.239712 27683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:18.239492 27676 sys_catalog.cc:455] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b70c49504e9a445cb0dd5d5acd1e06b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37181 } } }
I20260926 19:38:18.239939 27676 sys_catalog.cc:458] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:18.245503 27683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:18.253942 27683 catalog_manager.cc:1347] Generated new cluster ID: 4168db6f7cfb4b32941a9261d04ce9c0
I20260926 19:38:18.254065 27683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:18.272176 27683 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:18.272655 27683 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:18.282217 27683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b70c49504e9a445cb0dd5d5acd1e06b0: Generated new TSK 0
I20260926 19:38:18.283309 27683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:18.297231 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27597
W20260926 19:38:18.298672 27659 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38243
W20260926 19:38:18.299507 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27597: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:18.299690 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27597
2026-09-26T19:38:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1885 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:18 dist-test-slave-phbh krb5kdc[27695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:18 dist-test-slave-phbh krb5kdc[27695](info): set up 2 sockets
Sep 26 19:38:18 dist-test-slave-phbh krb5kdc[27695](info): commencing operation
krb5kdc: starting...
W20260926 19:38:19.449155 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:19 dist-test-slave-phbh krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:19.708009 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35519
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:20.036367 27708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:20.036677 27708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:20.036863 27708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:20.044252 27708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:20.044378 27708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:20.044482 27708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:20.044591 27708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:20.044675 27708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:20.054791 27708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:20.056125 27708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:20.057586 27708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:20.069433 27713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:20.069581 27714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:20.069605 27716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:20.070446 27708 server_base.cc:1034] running on GCE node
I20260926 19:38:20.071373 27708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:20.072932 27708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:20.074218 27708 hybrid_clock.cc:648] HybridClock initialized: now 1790451500074128 us; error 61 us; skew 500 ppm
I20260926 19:38:20.074589 27708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:20 dist-test-slave-phbh krb5kdc[27695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451500, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:20.084424 27708 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:20.086140 27708 webserver.cc:460] Webserver started at http://127.0.0.1:36933/ using document root <none> and password file <none>
I20260926 19:38:20.086805 27708 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:20.086906 27708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:20.087167 27708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:20.091295 27708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "411355f0ba244c44a6afaf3ee703a498"
format_stamp: "Formatted at 2026-09-26 19:38:20 on dist-test-slave-phbh"
server_key: "c2b625c0d2a172ebad7a5f1b110e917e"
server_key_iv: "afacb2fafe20f2c9d0536f11a5861f6a"
server_key_version: "encryptionkey@0"
I20260926 19:38:20.092101 27708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "411355f0ba244c44a6afaf3ee703a498"
format_stamp: "Formatted at 2026-09-26 19:38:20 on dist-test-slave-phbh"
server_key: "c2b625c0d2a172ebad7a5f1b110e917e"
server_key_iv: "afacb2fafe20f2c9d0536f11a5861f6a"
server_key_version: "encryptionkey@0"
I20260926 19:38:20.097546 27708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 19:38:20.101117 27723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:20.102490 27708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:38:20.102639 27708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "411355f0ba244c44a6afaf3ee703a498"
format_stamp: "Formatted at 2026-09-26 19:38:20 on dist-test-slave-phbh"
server_key: "c2b625c0d2a172ebad7a5f1b110e917e"
server_key_iv: "afacb2fafe20f2c9d0536f11a5861f6a"
server_key_version: "encryptionkey@0"
I20260926 19:38:20.102806 27708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:20.145399 27708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:20.149845 27708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:20.150080 27708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:20.168606 27708 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35519
I20260926 19:38:20.168637 27774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35519 every 8 connection(s)
I20260926 19:38:20.170007 27708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:20.173147 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27708
I20260926 19:38:20.173347 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:20.173900 7007 external_mini_cluster.cc:1428] Setting key e89c0feaf88b58c1875075313b24bb54
I20260926 19:38:20.174072 27775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:20.189777 27775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498: Bootstrap starting.
Sep 26 19:38:20 dist-test-slave-phbh krb5kdc[27695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 19:38:20.193902 27775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:20.195331 27775 log.cc:824] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:20.198997 27775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498: No bootstrap required, opened a new log
I20260926 19:38:20.202472 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:20.175708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39063 (local address 127.0.0.1:35519)
0926 19:38:20.176492 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:20.176506 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:20.176549 (+ 43us) server_negotiation.cc:407] Connection header received
0926 19:38:20.178650 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:20.178672 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:20.179070 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:20.179385 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:20.181233 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:20.183739 (+ 2506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:20.185864 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:20.186985 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:20.187185 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:20.192658 (+ 5473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:20.192669 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:20.192686 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:20.192732 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:20.196739 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:20.197675 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:20.197684 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:20.197691 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:20.197904 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:20.198380 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:20.198388 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:20.198394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:20.199056 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:20.199355 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:20.200616 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0926 19:38:20.201120 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":372,"threads_started":1}
I20260926 19:38:20.207021 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:20.209430 27775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "411355f0ba244c44a6afaf3ee703a498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260926 19:38:20.209913 27775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:20.210019 27775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 411355f0ba244c44a6afaf3ee703a498, State: Initialized, Role: FOLLOWER
I20260926 19:38:20.210709 27775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "411355f0ba244c44a6afaf3ee703a498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260926 19:38:20.211050 27775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:20.211220 27775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:20.211396 27775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:20.214689 27775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "411355f0ba244c44a6afaf3ee703a498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260926 19:38:20.215310 27775 leader_election.cc:302] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 411355f0ba244c44a6afaf3ee703a498; no voters:
I20260926 19:38:20.216007 27775 leader_election.cc:288] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:20.216200 27785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:20.217427 27785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [term 1 LEADER]: Becoming Leader. State: Replica: 411355f0ba244c44a6afaf3ee703a498, State: Running, Role: LEADER
I20260926 19:38:20.218058 27785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "411355f0ba244c44a6afaf3ee703a498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260926 19:38:20.218355 27775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:20.227988 27787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 411355f0ba244c44a6afaf3ee703a498. Latest consensus state: current_term: 1 leader_uuid: "411355f0ba244c44a6afaf3ee703a498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "411355f0ba244c44a6afaf3ee703a498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } } }
I20260926 19:38:20.227965 27786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "411355f0ba244c44a6afaf3ee703a498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "411355f0ba244c44a6afaf3ee703a498" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } } }
I20260926 19:38:20.228456 27787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:20.228456 27786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:20.230985 27793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:20.234985 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:20.212438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40119 (local address 127.0.0.1:35519)
0926 19:38:20.212629 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:20.212635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:20.213127 (+ 492us) server_negotiation.cc:407] Connection header received
0926 19:38:20.213338 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:20.213343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:20.213418 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:20.213519 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:20.215607 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:20.216903 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:20.218696 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:20.219471 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:20.219623 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:20.223895 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:20.223906 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:20.223914 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:20.223968 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:20.228795 (+ 4827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:20.229904 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:20.229913 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:20.229919 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:20.230123 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:20.230654 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:20.230659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:20.230665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:20.231028 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:20.231297 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:20.234160 (+ 2863us) server_negotiation.cc:299] Negotiation successful
0926 19:38:20.234483 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 19:38:20.246109 27793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:20.255513 27793 catalog_manager.cc:1347] Generated new cluster ID: 8bd84872af4745358933468ab31860ec
I20260926 19:38:20.255662 27793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:20.260728 27803 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:38:20.274545 27793 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:20.275820 27793 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:20.283023 27793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 411355f0ba244c44a6afaf3ee703a498: Generated new TSK 0
I20260926 19:38:20.283928 27793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:20.297117 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27708
I20260926 19:38:20.298228 27770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:20.388962 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27708
2026-09-26T19:38:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2090 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:20 dist-test-slave-phbh krb5kdc[27807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:20 dist-test-slave-phbh krb5kdc[27807](info): set up 2 sockets
Sep 26 19:38:20 dist-test-slave-phbh krb5kdc[27807](info): commencing operation
krb5kdc: starting...
W20260926 19:38:21.493214 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:21 dist-test-slave-phbh krb5kdc[27807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:21.747678 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:22.075508 27820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:22.075816 27820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:22.076007 27820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:22.083482 27820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:22.083603 27820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:22.083704 27820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:22.083815 27820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:22.083895 27820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:22.094199 27820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:22.095604 27820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:22.097406 27820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:22.109877 27826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:22.109946 27828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:22.109942 27825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:22.110833 27820 server_base.cc:1034] running on GCE node
I20260926 19:38:22.111606 27820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:22.113070 27820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:22.114267 27820 hybrid_clock.cc:648] HybridClock initialized: now 1790451502114202 us; error 56 us; skew 500 ppm
I20260926 19:38:22.114624 27820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:22 dist-test-slave-phbh krb5kdc[27807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451502, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:22.117827 27820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:22.119293 27820 webserver.cc:460] Webserver started at http://127.0.0.1:42299/ using document root <none> and password file <none>
I20260926 19:38:22.119925 27820 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:22.120026 27820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:22.120280 27820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:22.123323 27820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee"
format_stamp: "Formatted at 2026-09-26 19:38:22 on dist-test-slave-phbh"
server_key: "ff28316c3c1d37f71b6e6a72da574731"
server_key_iv: "192896a0ed347365fec44cebe3046081"
server_key_version: "encryptionkey@0"
I20260926 19:38:22.124008 27820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee"
format_stamp: "Formatted at 2026-09-26 19:38:22 on dist-test-slave-phbh"
server_key: "ff28316c3c1d37f71b6e6a72da574731"
server_key_iv: "192896a0ed347365fec44cebe3046081"
server_key_version: "encryptionkey@0"
I20260926 19:38:22.129226 27820 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260926 19:38:22.132758 27835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:22.134164 27820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:22.134349 27820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee"
format_stamp: "Formatted at 2026-09-26 19:38:22 on dist-test-slave-phbh"
server_key: "ff28316c3c1d37f71b6e6a72da574731"
server_key_iv: "192896a0ed347365fec44cebe3046081"
server_key_version: "encryptionkey@0"
I20260926 19:38:22.134552 27820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:22.151046 27820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:22.155162 27820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:22.155458 27820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:22.174196 27820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40831
I20260926 19:38:22.174234 27886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40831 every 8 connection(s)
I20260926 19:38:22.175606 27820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:22.179396 27887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:22.181820 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27820
I20260926 19:38:22.182020 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:22.182435 7007 external_mini_cluster.cc:1428] Setting key d5021b4616371ddd31444058f07d6d1b
I20260926 19:38:22.192816 27887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee: Bootstrap starting.
I20260926 19:38:22.196877 27887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:22.197889 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:22.184299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59995 (local address 127.0.0.1:40831)
0926 19:38:22.185238 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:22.185252 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:22.185302 (+ 50us) server_negotiation.cc:407] Connection header received
0926 19:38:22.187623 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:22.187653 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:22.188165 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:22.188601 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:22.189140 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:22.191221 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:22.192640 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:22.193700 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:22.193866 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:22.195045 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:22.195056 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:22.195074 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:22.195787 (+ 713us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:22.195893 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:22.195931 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:22.196056 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:22.196237 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 19:38:22.196502 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":462,"threads_started":1}
I20260926 19:38:22.198259 27887 log.cc:824] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:22.201674 27887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee: No bootstrap required, opened a new log
I20260926 19:38:22.202019 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:22.210611 27887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260926 19:38:22.211159 27887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:22.211301 27887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ae0bc68fbb4b6ab50430cd48dbcbee, State: Initialized, Role: FOLLOWER
I20260926 19:38:22.211999 27887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260926 19:38:22.212280 27887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:22.212446 27887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:22.212628 27887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:22.213228 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:22.206324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33137 (local address 127.0.0.1:40831)
0926 19:38:22.206485 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:22.206492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:22.206947 (+ 455us) server_negotiation.cc:407] Connection header received
0926 19:38:22.207075 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:22.207081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:22.207166 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:22.207274 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:22.207764 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:22.208931 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:22.210331 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:22.210968 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:22.211121 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:22.212364 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:22.212370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:22.212374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:22.212397 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:22.212454 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:22.212479 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:22.212576 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:22.212738 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 19:38:22.212879 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 19:38:22.215492 27887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260926 19:38:22.215938 27887 leader_election.cc:302] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35ae0bc68fbb4b6ab50430cd48dbcbee; no voters:
I20260926 19:38:22.216465 27887 leader_election.cc:288] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:22.216590 27897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:22.217386 27897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [term 1 LEADER]: Becoming Leader. State: Replica: 35ae0bc68fbb4b6ab50430cd48dbcbee, State: Running, Role: LEADER
I20260926 19:38:22.217891 27897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } }
I20260926 19:38:22.218669 27887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:22.220671 27899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ae0bc68fbb4b6ab50430cd48dbcbee. Latest consensus state: current_term: 1 leader_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } } }
I20260926 19:38:22.221108 27899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:22.222360 27898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ae0bc68fbb4b6ab50430cd48dbcbee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40831 } } }
I20260926 19:38:22.222796 27898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:22.223109 27904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:22.228711 27904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:22.237524 27904 catalog_manager.cc:1347] Generated new cluster ID: 9f606dc664994129ae05fce3cdba1d6a
I20260926 19:38:22.237664 27904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:22.251240 27904 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:22.252549 27904 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:22.260121 27904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35ae0bc68fbb4b6ab50430cd48dbcbee: Generated new TSK 0
I20260926 19:38:22.261010 27904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:22.273818 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27820
W20260926 19:38:22.274999 27882 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59995
W20260926 19:38:22.275748 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27820: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:22.276005 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27820
2026-09-26T19:38:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1885 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:22 dist-test-slave-phbh krb5kdc[27918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:22 dist-test-slave-phbh krb5kdc[27918](info): set up 2 sockets
Sep 26 19:38:22 dist-test-slave-phbh krb5kdc[27918](info): commencing operation
krb5kdc: starting...
W20260926 19:38:23.385231 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.011s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:23 dist-test-slave-phbh krb5kdc[27918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:23.646471 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:23.972057 27931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:23.972350 27931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:23.972513 27931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:23.979871 27931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:23.979983 27931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:23.980089 27931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:23.980168 27931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:23.980223 27931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:23.990200 27931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:23.991482 27931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:23.992723 27931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:24.004686 27936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:24.004608 27939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:24.004551 27937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:24.005774 27931 server_base.cc:1034] running on GCE node
I20260926 19:38:24.006652 27931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:24.008054 27931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:24.009294 27931 hybrid_clock.cc:648] HybridClock initialized: now 1790451504009246 us; error 35 us; skew 500 ppm
I20260926 19:38:24.009665 27931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:24 dist-test-slave-phbh krb5kdc[27918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451504, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:24.017776 27931 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:24.019361 27931 webserver.cc:460] Webserver started at http://127.0.0.1:33911/ using document root <none> and password file <none>
I20260926 19:38:24.020027 27931 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:24.020131 27931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:24.020385 27931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:24.023464 27931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31"
format_stamp: "Formatted at 2026-09-26 19:38:24 on dist-test-slave-phbh"
server_key: "6eb323f46e659c80957907e0798ee0d5"
server_key_iv: "b7fcfac731fb287135b9ef1080ec9400"
server_key_version: "encryptionkey@0"
I20260926 19:38:24.024158 27931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31"
format_stamp: "Formatted at 2026-09-26 19:38:24 on dist-test-slave-phbh"
server_key: "6eb323f46e659c80957907e0798ee0d5"
server_key_iv: "b7fcfac731fb287135b9ef1080ec9400"
server_key_version: "encryptionkey@0"
I20260926 19:38:24.029325 27931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 19:38:24.032760 27946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:24.034112 27931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:24.034288 27931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31"
format_stamp: "Formatted at 2026-09-26 19:38:24 on dist-test-slave-phbh"
server_key: "6eb323f46e659c80957907e0798ee0d5"
server_key_iv: "b7fcfac731fb287135b9ef1080ec9400"
server_key_version: "encryptionkey@0"
I20260926 19:38:24.034461 27931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:24.040138 27931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:24.044132 27931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:24.044376 27931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:24.063441 27931 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37579
I20260926 19:38:24.063480 27997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37579 every 8 connection(s)
I20260926 19:38:24.064883 27931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:24.068636 27998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:24.071106 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 27931
I20260926 19:38:24.071308 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:24.071833 7007 external_mini_cluster.cc:1428] Setting key 449909de444fb6aabf532dca53a4caff
I20260926 19:38:24.079681 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:24.073704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40089 (local address 127.0.0.1:37579)
0926 19:38:24.074547 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:24.074558 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:24.074607 (+ 49us) server_negotiation.cc:407] Connection header received
0926 19:38:24.076233 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:24.076256 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:24.076636 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:24.077023 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:24.078103 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:24.078109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:24.078122 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:24.078135 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:24.078230 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:24.078252 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:24.078431 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:24.078533 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 19:38:24.078739 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":391,"threads_started":1}
I20260926 19:38:24.081315 27998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31: Bootstrap starting.
I20260926 19:38:24.083501 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:24.084576 27998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:24.085613 27998 log.cc:824] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:24.088567 27998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31: No bootstrap required, opened a new log
I20260926 19:38:24.090340 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:24.087762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35173 (local address 127.0.0.1:37579)
0926 19:38:24.087940 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:24.087946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:24.088408 (+ 462us) server_negotiation.cc:407] Connection header received
0926 19:38:24.088533 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:24.088541 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:24.088608 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:24.088718 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:24.089691 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:24.089696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:24.089701 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:24.089720 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:24.089764 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:24.089784 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:24.089906 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:24.089978 (+ 72us) server_negotiation.cc:299] Negotiation successful
0926 19:38:24.090034 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:38:24.096612 27998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260926 19:38:24.097014 27998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:24.097134 27998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b6b6b4bd5924324a3cdd0326d3c8b31, State: Initialized, Role: FOLLOWER
I20260926 19:38:24.097707 27998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260926 19:38:24.097970 27998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:24.098127 27998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:24.098294 27998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:24.100903 27998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260926 19:38:24.101398 27998 leader_election.cc:302] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b6b6b4bd5924324a3cdd0326d3c8b31; no voters:
I20260926 19:38:24.102099 27998 leader_election.cc:288] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:24.102241 28008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:24.102943 28008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [term 1 LEADER]: Becoming Leader. State: Replica: 8b6b6b4bd5924324a3cdd0326d3c8b31, State: Running, Role: LEADER
I20260926 19:38:24.103560 28008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } }
I20260926 19:38:24.104425 27998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:24.108043 28010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b6b6b4bd5924324a3cdd0326d3c8b31. Latest consensus state: current_term: 1 leader_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } } }
I20260926 19:38:24.108448 28010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:24.108964 28009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b6b6b4bd5924324a3cdd0326d3c8b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37579 } } }
I20260926 19:38:24.109287 28009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:24.111852 28015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:24.117087 28015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:24.127022 28015 catalog_manager.cc:1347] Generated new cluster ID: 8a387fa2271e4f1dba7e0706af139514
I20260926 19:38:24.127140 28015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:24.141453 28015 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:24.141927 28015 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:24.165481 28015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b6b6b4bd5924324a3cdd0326d3c8b31: Generated new TSK 0
I20260926 19:38:24.166631 28015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:24.183940 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27931
W20260926 19:38:24.186069 27993 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40089
W20260926 19:38:24.186818 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 27931: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:24.186980 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 27931
2026-09-26T19:38:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1908 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:24 dist-test-slave-phbh krb5kdc[28029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:24 dist-test-slave-phbh krb5kdc[28029](info): set up 2 sockets
Sep 26 19:38:24 dist-test-slave-phbh krb5kdc[28029](info): commencing operation
krb5kdc: starting...
W20260926 19:38:25.313194 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:25 dist-test-slave-phbh krb5kdc[28029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:25.584612 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:25.915395 28042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:25.915681 28042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:25.915865 28042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:25.923190 28042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:25.923331 28042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:25.923439 28042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:25.923565 28042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:25.923646 28042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:25.933492 28042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:25.934783 28042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:25.936102 28042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:25.949793 28047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:25.949718 28050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:25.949767 28048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:25.950918 28042 server_base.cc:1034] running on GCE node
I20260926 19:38:25.951732 28042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:25.953167 28042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:25.954401 28042 hybrid_clock.cc:648] HybridClock initialized: now 1790451505954366 us; error 50 us; skew 500 ppm
I20260926 19:38:25.954756 28042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:25 dist-test-slave-phbh krb5kdc[28029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:25.957933 28042 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:25.959468 28042 webserver.cc:460] Webserver started at http://127.0.0.1:36113/ using document root <none> and password file <none>
I20260926 19:38:25.960121 28042 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:25.960216 28042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:25.960465 28042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:25.963495 28042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "f561225a618f460891f5e4fd2a6bbceb"
format_stamp: "Formatted at 2026-09-26 19:38:25 on dist-test-slave-phbh"
server_key: "3dd82df5256c7e4e76017635e261c17b"
server_key_iv: "0e68bb48a295c418d4397d1c26f80be6"
server_key_version: "encryptionkey@0"
I20260926 19:38:25.964346 28042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "f561225a618f460891f5e4fd2a6bbceb"
format_stamp: "Formatted at 2026-09-26 19:38:25 on dist-test-slave-phbh"
server_key: "3dd82df5256c7e4e76017635e261c17b"
server_key_iv: "0e68bb48a295c418d4397d1c26f80be6"
server_key_version: "encryptionkey@0"
I20260926 19:38:25.969939 28042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 19:38:25.973399 28057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:25.974749 28042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 19:38:25.974884 28042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "f561225a618f460891f5e4fd2a6bbceb"
format_stamp: "Formatted at 2026-09-26 19:38:25 on dist-test-slave-phbh"
server_key: "3dd82df5256c7e4e76017635e261c17b"
server_key_iv: "0e68bb48a295c418d4397d1c26f80be6"
server_key_version: "encryptionkey@0"
I20260926 19:38:25.975044 28042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:25.999577 28042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:26.003865 28042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:26.004122 28042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:26.022568 28042 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37639
I20260926 19:38:26.022611 28108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37639 every 8 connection(s)
I20260926 19:38:26.023940 28042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:26.027711 28109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:26.030920 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28042
I20260926 19:38:26.031116 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:26.031651 7007 external_mini_cluster.cc:1428] Setting key 17f207df0f4654645c2b5c1fc84beb51
I20260926 19:38:26.040588 28109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb: Bootstrap starting.
I20260926 19:38:26.044564 28109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:26.045992 28109 log.cc:824] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb: Log is configured to *not* fsync() on all Append() calls
Sep 26 19:38:26 dist-test-slave-phbh krb5kdc[28029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 19:38:26.049226 28109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb: No bootstrap required, opened a new log
I20260926 19:38:26.056254 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:26.033654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:40595 (local address 127.0.0.1:37639)
0926 19:38:26.034480 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:26.034492 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:26.034534 (+ 42us) server_negotiation.cc:407] Connection header received
0926 19:38:26.036379 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:26.036403 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:26.036925 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:26.037295 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:26.039075 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:26.040448 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:26.042348 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:26.043022 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:26.043301 (+ 279us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:26.048453 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:26.048461 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:26.048477 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:26.048515 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:26.051914 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:26.052819 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:26.052829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:26.052842 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:26.053006 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:26.053493 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:26.053497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:26.053499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:26.054039 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:26.054385 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:26.054895 (+ 510us) server_negotiation.cc:299] Negotiation successful
0926 19:38:26.055216 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":390,"threads_started":1}
I20260926 19:38:26.057585 28109 raft_consensus.cc:348] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f561225a618f460891f5e4fd2a6bbceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260926 19:38:26.057965 28109 raft_consensus.cc:374] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:26.058070 28109 raft_consensus.cc:729] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f561225a618f460891f5e4fd2a6bbceb, State: Initialized, Role: FOLLOWER
I20260926 19:38:26.058696 28109 consensus_queue.cc:260] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f561225a618f460891f5e4fd2a6bbceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260926 19:38:26.058983 28109 raft_consensus.cc:388] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:26.059129 28109 raft_consensus.cc:482] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:26.059296 28109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:26.060384 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:26.062428 28109 raft_consensus.cc:504] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f561225a618f460891f5e4fd2a6bbceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260926 19:38:26.063030 28109 leader_election.cc:302] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f561225a618f460891f5e4fd2a6bbceb; no voters:
I20260926 19:38:26.063742 28109 leader_election.cc:288] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:26.063889 28114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:26.065590 28114 raft_consensus.cc:686] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [term 1 LEADER]: Becoming Leader. State: Replica: f561225a618f460891f5e4fd2a6bbceb, State: Running, Role: LEADER
I20260926 19:38:26.066314 28114 consensus_queue.cc:237] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f561225a618f460891f5e4fd2a6bbceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260926 19:38:26.066596 28109 sys_catalog.cc:564] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:26.070525 28117 sys_catalog.cc:455] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f561225a618f460891f5e4fd2a6bbceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f561225a618f460891f5e4fd2a6bbceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } } }
I20260926 19:38:26.070947 28117 sys_catalog.cc:458] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:26.071542 28119 sys_catalog.cc:455] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f561225a618f460891f5e4fd2a6bbceb. Latest consensus state: current_term: 1 leader_uuid: "f561225a618f460891f5e4fd2a6bbceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f561225a618f460891f5e4fd2a6bbceb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } } }
I20260926 19:38:26.071875 28119 sys_catalog.cc:458] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:26.074746 28125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:26.082289 28125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:26.086865 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:26.067667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:47447 (local address 127.0.0.1:37639)
0926 19:38:26.067843 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:26.067851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:26.068866 (+ 1015us) server_negotiation.cc:407] Connection header received
0926 19:38:26.069194 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:26.069203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:26.069327 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:26.069427 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:26.071694 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:26.073009 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:26.075073 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:26.075538 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:26.075699 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:26.079097 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:26.079106 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:26.079111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:26.079160 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:26.083097 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:26.084099 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:26.084104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:26.084108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:26.084262 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:26.084889 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:26.084895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:26.084899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:26.085306 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:26.085545 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:26.086263 (+ 718us) server_negotiation.cc:299] Negotiation successful
0926 19:38:26.086451 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 19:38:26.096357 28125 catalog_manager.cc:1347] Generated new cluster ID: f087724bad6f49c9aa5ead5cf6957a60
I20260926 19:38:26.096480 28125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:26.113376 28125 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:26.115509 28125 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:26.125870 28125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f561225a618f460891f5e4fd2a6bbceb: Generated new TSK 0
I20260926 19:38:26.126734 28125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:26.138082 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28042
I20260926 19:38:26.139360 28104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:26.231482 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28042
W20260926 19:38:26.248647 28027 connection.cc:535] client connection to 127.0.0.1:37639 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37639): Connection reset by peer (error 104)
2026-09-26T19:38:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2047 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:26 dist-test-slave-phbh krb5kdc[28142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:26 dist-test-slave-phbh krb5kdc[28142](info): set up 2 sockets
Sep 26 19:38:26 dist-test-slave-phbh krb5kdc[28142](info): commencing operation
krb5kdc: starting...
W20260926 19:38:27.337829 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:27 dist-test-slave-phbh krb5kdc[28142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:27.613318 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:27.941006 28155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:27.941324 28155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:27.941515 28155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:27.948653 28155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:27.948844 28155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:27.948958 28155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:27.949081 28155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:27.949170 28155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:27.959285 28155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:27.960611 28155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:27.961992 28155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:27.974259 28161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:27.974323 28163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:27.974232 28160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:27.975587 28155 server_base.cc:1034] running on GCE node
I20260926 19:38:27.976446 28155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:27.978081 28155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:27.979337 28155 hybrid_clock.cc:648] HybridClock initialized: now 1790451507979303 us; error 52 us; skew 500 ppm
I20260926 19:38:27.979696 28155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:27 dist-test-slave-phbh krb5kdc[28142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:27.983108 28155 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:27.984702 28155 webserver.cc:460] Webserver started at http://127.0.0.1:42607/ using document root <none> and password file <none>
I20260926 19:38:27.985455 28155 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:27.985607 28155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:27.985920 28155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:27.989132 28155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "08b9986b376f463295f00b38c8fe62b5"
format_stamp: "Formatted at 2026-09-26 19:38:27 on dist-test-slave-phbh"
server_key: "dd4786530109c58504321b2d00844519"
server_key_iv: "1140a7223e6015efff9cd500d581b6f9"
server_key_version: "encryptionkey@0"
I20260926 19:38:27.989928 28155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "08b9986b376f463295f00b38c8fe62b5"
format_stamp: "Formatted at 2026-09-26 19:38:27 on dist-test-slave-phbh"
server_key: "dd4786530109c58504321b2d00844519"
server_key_iv: "1140a7223e6015efff9cd500d581b6f9"
server_key_version: "encryptionkey@0"
I20260926 19:38:27.995348 28155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:27.999378 28170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:28.001118 28155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:28.001338 28155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "08b9986b376f463295f00b38c8fe62b5"
format_stamp: "Formatted at 2026-09-26 19:38:27 on dist-test-slave-phbh"
server_key: "dd4786530109c58504321b2d00844519"
server_key_iv: "1140a7223e6015efff9cd500d581b6f9"
server_key_version: "encryptionkey@0"
I20260926 19:38:28.001564 28155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:28.017583 28155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:28.022296 28155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:28.022606 28155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:28.042816 28155 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46231
I20260926 19:38:28.042846 28221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46231 every 8 connection(s)
I20260926 19:38:28.044341 28155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:28.048528 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:28.049021 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28155
I20260926 19:38:28.049227 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:28.049654 7007 external_mini_cluster.cc:1428] Setting key f76dac792b23efaf2e1831072aae6f33
I20260926 19:38:28.062892 28222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5: Bootstrap starting.
I20260926 19:38:28.065138 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:28.051761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:59509 (local address 127.0.0.1:46231)
0926 19:38:28.052612 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:28.052633 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:28.052683 (+ 50us) server_negotiation.cc:407] Connection header received
0926 19:38:28.054865 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:28.054894 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:28.055383 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:28.055767 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:28.056458 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:28.057838 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:28.059816 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:28.060621 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:28.060868 (+ 247us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:28.062304 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:28.062312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:28.062328 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:28.062901 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:28.063006 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:28.063076 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:28.063301 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:28.063710 (+ 409us) server_negotiation.cc:299] Negotiation successful
0926 19:38:28.063920 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":373,"threads_started":1}
I20260926 19:38:28.066807 28222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:28.068084 28222 log.cc:824] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:28.069979 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:28.071367 28222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5: No bootstrap required, opened a new log
I20260926 19:38:28.080345 28222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9986b376f463295f00b38c8fe62b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260926 19:38:28.080758 28222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:28.080950 28222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b9986b376f463295f00b38c8fe62b5, State: Initialized, Role: FOLLOWER
I20260926 19:38:28.081805 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:28.074837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:54741 (local address 127.0.0.1:46231)
0926 19:38:28.075019 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:28.075025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:28.075423 (+ 398us) server_negotiation.cc:407] Connection header received
0926 19:38:28.075589 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:28.075595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:28.075690 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:28.075815 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:28.076270 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:28.077501 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:28.078884 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:28.079552 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:28.079711 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:28.080882 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:28.080887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:28.080890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:28.080933 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:28.081000 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:28.081017 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:28.081151 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:28.081411 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 19:38:28.081477 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 19:38:28.081696 28222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9986b376f463295f00b38c8fe62b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260926 19:38:28.081997 28222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:28.082165 28222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:28.082350 28222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:28.084990 28222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9986b376f463295f00b38c8fe62b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260926 19:38:28.085608 28222 leader_election.cc:302] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08b9986b376f463295f00b38c8fe62b5; no voters:
I20260926 19:38:28.086243 28222 leader_election.cc:288] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:28.086453 28232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:28.087203 28232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [term 1 LEADER]: Becoming Leader. State: Replica: 08b9986b376f463295f00b38c8fe62b5, State: Running, Role: LEADER
I20260926 19:38:28.087765 28232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9986b376f463295f00b38c8fe62b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260926 19:38:28.088379 28222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:28.091064 28234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08b9986b376f463295f00b38c8fe62b5. Latest consensus state: current_term: 1 leader_uuid: "08b9986b376f463295f00b38c8fe62b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9986b376f463295f00b38c8fe62b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } } }
I20260926 19:38:28.091550 28234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:28.091320 28233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08b9986b376f463295f00b38c8fe62b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08b9986b376f463295f00b38c8fe62b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } } }
I20260926 19:38:28.091734 28233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:28.095242 28238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:28.101797 28238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:28.110913 28238 catalog_manager.cc:1347] Generated new cluster ID: b0cb06c3b21146f4b921353de520ef4d
I20260926 19:38:28.111065 28238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:28.132220 28238 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:28.134212 28238 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:28.141659 28238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08b9986b376f463295f00b38c8fe62b5: Generated new TSK 0
I20260926 19:38:28.142757 28238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:28.149941 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28155
W20260926 19:38:28.151156 28217 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.167:59509
W20260926 19:38:28.151942 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28155: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:28.152089 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28155
W20260926 19:38:28.169083 28140 connection.cc:535] client connection to 127.0.0.1:46231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46231): Connection reset by peer (error 104)
2026-09-26T19:38:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1920 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:28 dist-test-slave-phbh krb5kdc[28253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:28 dist-test-slave-phbh krb5kdc[28253](info): set up 2 sockets
Sep 26 19:38:28 dist-test-slave-phbh krb5kdc[28253](info): commencing operation
krb5kdc: starting...
W20260926 19:38:29.281360 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:29 dist-test-slave-phbh krb5kdc[28253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:29.540287 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:29.868224 28266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:29.868522 28266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:29.868712 28266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:29.875934 28266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:29.876068 28266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:29.876153 28266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:29.876255 28266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:29.876341 28266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:29.886276 28266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:29.887665 28266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:29.889117 28266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:29.900588 28272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:29.900681 28271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:29.900681 28274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:29.901561 28266 server_base.cc:1034] running on GCE node
I20260926 19:38:29.902240 28266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:29.903590 28266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:29.904839 28266 hybrid_clock.cc:648] HybridClock initialized: now 1790451509904749 us; error 37 us; skew 500 ppm
I20260926 19:38:29.905156 28266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:29 dist-test-slave-phbh krb5kdc[28253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451509, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:29.911968 28266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:29.913692 28266 webserver.cc:460] Webserver started at http://127.0.0.1:45391/ using document root <none> and password file <none>
I20260926 19:38:29.914422 28266 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:29.914572 28266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:29.914889 28266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:29.917961 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "b0912a9e7574477aa254f8f051072487"
format_stamp: "Formatted at 2026-09-26 19:38:29 on dist-test-slave-phbh"
server_key: "c8b856c191f454230cc561c3ef7fd5b0"
server_key_iv: "52e78eb07311280942b3054bdc4bb1ef"
server_key_version: "encryptionkey@0"
I20260926 19:38:29.918747 28266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "b0912a9e7574477aa254f8f051072487"
format_stamp: "Formatted at 2026-09-26 19:38:29 on dist-test-slave-phbh"
server_key: "c8b856c191f454230cc561c3ef7fd5b0"
server_key_iv: "52e78eb07311280942b3054bdc4bb1ef"
server_key_version: "encryptionkey@0"
I20260926 19:38:29.923964 28266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:38:29.927353 28281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:29.928598 28266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 19:38:29.928846 28266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "b0912a9e7574477aa254f8f051072487"
format_stamp: "Formatted at 2026-09-26 19:38:29 on dist-test-slave-phbh"
server_key: "c8b856c191f454230cc561c3ef7fd5b0"
server_key_iv: "52e78eb07311280942b3054bdc4bb1ef"
server_key_version: "encryptionkey@0"
I20260926 19:38:29.929054 28266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:29.934839 28266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:29.938902 28266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:29.939204 28266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:29.957985 28266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45821
I20260926 19:38:29.958029 28332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45821 every 8 connection(s)
I20260926 19:38:29.959429 28266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:29.963541 28333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:29.964145 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28266
I20260926 19:38:29.964349 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:29.964849 7007 external_mini_cluster.cc:1428] Setting key e2927cebbbde7e0926ef4be9c555ff9a
I20260926 19:38:29.973754 28336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:29.966972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:52803 (local address 127.0.0.1:45821)
0926 19:38:29.967943 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:29.967959 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:29.968339 (+ 380us) server_negotiation.cc:407] Connection header received
0926 19:38:29.970123 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:29.970150 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:29.970596 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:29.971047 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:29.971957 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:29.971966 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:29.971981 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:29.971996 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:29.972055 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:29.972078 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:29.972280 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:29.972400 (+ 120us) server_negotiation.cc:299] Negotiation successful
0926 19:38:29.972580 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":406,"threads_started":1}
I20260926 19:38:29.977322 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:29.977597 28333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487: Bootstrap starting.
I20260926 19:38:29.980545 28333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:29.981742 28333 log.cc:824] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:29.984193 28336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:29.981595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:41041 (local address 127.0.0.1:45821)
0926 19:38:29.981792 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:29.981798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:29.982244 (+ 446us) server_negotiation.cc:407] Connection header received
0926 19:38:29.982436 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:29.982443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:29.982500 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:29.982604 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:29.983554 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:29.983559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:29.983564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:29.983586 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:29.983630 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:29.983645 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:29.983761 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:29.983853 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 19:38:29.983914 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:38:29.984529 28333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487: No bootstrap required, opened a new log
I20260926 19:38:29.992306 28333 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0912a9e7574477aa254f8f051072487" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260926 19:38:29.992694 28333 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:29.992835 28333 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0912a9e7574477aa254f8f051072487, State: Initialized, Role: FOLLOWER
I20260926 19:38:29.993391 28333 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0912a9e7574477aa254f8f051072487" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260926 19:38:29.993635 28333 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:29.993779 28333 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:29.993942 28333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:29.996212 28333 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0912a9e7574477aa254f8f051072487" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260926 19:38:29.996696 28333 leader_election.cc:302] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0912a9e7574477aa254f8f051072487; no voters:
I20260926 19:38:29.997319 28333 leader_election.cc:288] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:29.997458 28343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:29.998137 28343 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [term 1 LEADER]: Becoming Leader. State: Replica: b0912a9e7574477aa254f8f051072487, State: Running, Role: LEADER
I20260926 19:38:29.998633 28343 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0912a9e7574477aa254f8f051072487" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } }
I20260926 19:38:29.999349 28333 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:30.001379 28344 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0912a9e7574477aa254f8f051072487" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0912a9e7574477aa254f8f051072487" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } } }
I20260926 19:38:30.001899 28344 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:30.003094 28351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:30.002509 28345 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0912a9e7574477aa254f8f051072487. Latest consensus state: current_term: 1 leader_uuid: "b0912a9e7574477aa254f8f051072487" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0912a9e7574477aa254f8f051072487" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45821 } } }
I20260926 19:38:30.003911 28345 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:30.008757 28351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:30.018056 28351 catalog_manager.cc:1347] Generated new cluster ID: 7eba066f413346d686b66f7a8c9f5c53
I20260926 19:38:30.018217 28351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:30.053715 28351 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:30.054178 28351 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:30.062002 28351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0912a9e7574477aa254f8f051072487: Generated new TSK 0
I20260926 19:38:30.062849 28351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:30.071053 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28266
W20260926 19:38:30.072325 28328 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.167:52803
W20260926 19:38:30.073163 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28266: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:30.073321 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28266
2026-09-26T19:38:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1918 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:30 dist-test-slave-phbh krb5kdc[28364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:30 dist-test-slave-phbh krb5kdc[28364](info): set up 2 sockets
Sep 26 19:38:30 dist-test-slave-phbh krb5kdc[28364](info): commencing operation
krb5kdc: starting...
W20260926 19:38:31.189378 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:31 dist-test-slave-phbh krb5kdc[28364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:31.462190 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:31.797600 28377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:31.797957 28377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:31.798125 28377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:31.805531 28377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:31.805691 28377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:31.805792 28377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:31.805927 28377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:31.806005 28377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:31.816102 28377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41787
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:31.817530 28377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:31.818864 28377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:31.830909 28382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:31.830924 28383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:31.830717 28385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:31.831980 28377 server_base.cc:1034] running on GCE node
I20260926 19:38:31.832896 28377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:31.834331 28377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:31.835585 28377 hybrid_clock.cc:648] HybridClock initialized: now 1790451511835543 us; error 56 us; skew 500 ppm
I20260926 19:38:31.835935 28377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:31 dist-test-slave-phbh krb5kdc[28364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451511, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:31.839023 28377 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:31.840600 28377 webserver.cc:460] Webserver started at http://127.0.0.1:42947/ using document root <none> and password file <none>
I20260926 19:38:31.841367 28377 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:31.841514 28377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:31.841831 28377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:31.844820 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "a0db99c463694c64be195be3db862b7d"
format_stamp: "Formatted at 2026-09-26 19:38:31 on dist-test-slave-phbh"
server_key: "c5fde461642049e68013a84dc6e2cba7"
server_key_iv: "c2a3d9511456a7e8303b27085a855c58"
server_key_version: "encryptionkey@0"
I20260926 19:38:31.845616 28377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "a0db99c463694c64be195be3db862b7d"
format_stamp: "Formatted at 2026-09-26 19:38:31 on dist-test-slave-phbh"
server_key: "c5fde461642049e68013a84dc6e2cba7"
server_key_iv: "c2a3d9511456a7e8303b27085a855c58"
server_key_version: "encryptionkey@0"
I20260926 19:38:31.850718 28377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:31.854231 28392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:31.855623 28377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:38:31.855822 28377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "a0db99c463694c64be195be3db862b7d"
format_stamp: "Formatted at 2026-09-26 19:38:31 on dist-test-slave-phbh"
server_key: "c5fde461642049e68013a84dc6e2cba7"
server_key_iv: "c2a3d9511456a7e8303b27085a855c58"
server_key_version: "encryptionkey@0"
I20260926 19:38:31.856034 28377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:31.874068 28377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:31.878566 28377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:31.878888 28377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:31.897557 28377 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41787
I20260926 19:38:31.897612 28443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41787 every 8 connection(s)
I20260926 19:38:31.898942 28377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:31.902580 28444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:31.908097 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28377
I20260926 19:38:31.908342 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:31.908861 7007 external_mini_cluster.cc:1428] Setting key efd7ce4b4e0a63ccaa398267ecc8e18d
I20260926 19:38:31.913332 28444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d: Bootstrap starting.
I20260926 19:38:31.917006 28444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:31.918226 28444 log.cc:824] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:31.921407 28444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d: No bootstrap required, opened a new log
Sep 26 19:38:31 dist-test-slave-phbh krb5kdc[28364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 19:38:31.930024 28444 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0db99c463694c64be195be3db862b7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41787 } }
I20260926 19:38:31.930415 28444 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:31.930550 28444 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0db99c463694c64be195be3db862b7d, State: Initialized, Role: FOLLOWER
I20260926 19:38:31.931267 28444 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0db99c463694c64be195be3db862b7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41787 } }
I20260926 19:38:31.931552 28444 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:31.931696 28444 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:31.931881 28444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:31.933147 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:31.910846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:56531 (local address 127.0.0.1:41787)
0926 19:38:31.911669 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:31.911680 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:31.911717 (+ 37us) server_negotiation.cc:407] Connection header received
0926 19:38:31.913401 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:31.913422 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:31.913842 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:31.914137 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:31.916013 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:31.917442 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:31.919184 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:31.919967 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:31.920092 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:31.925042 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:31.925050 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:31.925066 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:31.925103 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:31.928277 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:31.929162 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:31.929168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:31.929175 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:31.929338 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:31.929791 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:31.929801 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:31.929803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:31.930388 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:31.930769 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:31.931292 (+ 523us) server_negotiation.cc:299] Negotiation successful
0926 19:38:31.931663 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":387,"threads_started":1}
I20260926 19:38:31.934698 28444 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0db99c463694c64be195be3db862b7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41787 } }
I20260926 19:38:31.935215 28444 leader_election.cc:302] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0db99c463694c64be195be3db862b7d; no voters:
I20260926 19:38:31.935920 28444 leader_election.cc:288] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:31.936023 28449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:31.937111 28449 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [term 1 LEADER]: Becoming Leader. State: Replica: a0db99c463694c64be195be3db862b7d, State: Running, Role: LEADER
I20260926 19:38:31.937206 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:31.937757 28449 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0db99c463694c64be195be3db862b7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41787 } }
I20260926 19:38:31.938243 28444 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:31.941115 28451 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0db99c463694c64be195be3db862b7d. Latest consensus state: current_term: 1 leader_uuid: "a0db99c463694c64be195be3db862b7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0db99c463694c64be195be3db862b7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41787 } } }
I20260926 19:38:31.941582 28451 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:31.946244 28450 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0db99c463694c64be195be3db862b7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0db99c463694c64be195be3db862b7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41787 } } }
I20260926 19:38:31.946636 28450 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:31.949823 28456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:31.955981 28456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:31.978181 28456 catalog_manager.cc:1347] Generated new cluster ID: 1b9d15c432d34017aa56e67a82b77323
I20260926 19:38:31.978312 28456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:31.979091 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:31.951872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:42013 (local address 127.0.0.1:41787)
0926 19:38:31.952070 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:31.952078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:31.959675 (+ 7597us) server_negotiation.cc:407] Connection header received
0926 19:38:31.959981 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:31.959989 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:31.960081 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:31.960203 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:31.962399 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:31.963483 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:31.966190 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:31.967125 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:31.967264 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:31.971825 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:31.971836 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:31.971844 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:31.971901 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:31.975327 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:31.976316 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:31.976321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:31.976325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:31.976480 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:31.977121 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:31.977126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:31.977129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:31.977565 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:31.977837 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:31.978554 (+ 717us) server_negotiation.cc:299] Negotiation successful
0926 19:38:31.978752 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 19:38:32.026520 28456 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:32.028684 28456 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:32.039723 28456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0db99c463694c64be195be3db862b7d: Generated new TSK 0
I20260926 19:38:32.040902 28456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:32.051703 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28377
I20260926 19:38:32.053011 28439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:32.151516 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28377
W20260926 19:38:32.168663 28362 connection.cc:535] client connection to 127.0.0.1:41787 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41787): Connection reset by peer (error 104)
2026-09-26T19:38:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2081 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:32 dist-test-slave-phbh krb5kdc[28476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:32 dist-test-slave-phbh krb5kdc[28476](info): set up 2 sockets
Sep 26 19:38:32 dist-test-slave-phbh krb5kdc[28476](info): commencing operation
krb5kdc: starting...
W20260926 19:38:33.277827 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:33 dist-test-slave-phbh krb5kdc[28476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:33.540177 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:33.869390 28489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:33.869688 28489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:33.869845 28489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:33.877121 28489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:33.877233 28489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:33.877334 28489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:33.877434 28489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:33.877503 28489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:33.887300 28489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:33.888584 28489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:33.889947 28489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:33.901803 28495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:33.901829 28497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:33.901876 28494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:33.903108 28489 server_base.cc:1034] running on GCE node
I20260926 19:38:33.903976 28489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:33.905485 28489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:33.906682 28489 hybrid_clock.cc:648] HybridClock initialized: now 1790451513906624 us; error 50 us; skew 500 ppm
I20260926 19:38:33.907025 28489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:33 dist-test-slave-phbh krb5kdc[28476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451513, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:33.910142 28489 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:33.911525 28489 webserver.cc:460] Webserver started at http://127.0.0.1:43359/ using document root <none> and password file <none>
I20260926 19:38:33.912145 28489 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:33.912240 28489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:33.912487 28489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:33.915540 28489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "dfbf1a35ca594666a0e17d56ea8965c9"
format_stamp: "Formatted at 2026-09-26 19:38:33 on dist-test-slave-phbh"
server_key: "76f2a7a82c0fc6ffdfe332424745654d"
server_key_iv: "cb320ac3a6ce4a1c8e07882ee5908e55"
server_key_version: "encryptionkey@0"
I20260926 19:38:33.916249 28489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "dfbf1a35ca594666a0e17d56ea8965c9"
format_stamp: "Formatted at 2026-09-26 19:38:33 on dist-test-slave-phbh"
server_key: "76f2a7a82c0fc6ffdfe332424745654d"
server_key_iv: "cb320ac3a6ce4a1c8e07882ee5908e55"
server_key_version: "encryptionkey@0"
I20260926 19:38:33.921450 28489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 19:38:33.924865 28504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:33.926219 28489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:38:33.926409 28489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "dfbf1a35ca594666a0e17d56ea8965c9"
format_stamp: "Formatted at 2026-09-26 19:38:33 on dist-test-slave-phbh"
server_key: "76f2a7a82c0fc6ffdfe332424745654d"
server_key_iv: "cb320ac3a6ce4a1c8e07882ee5908e55"
server_key_version: "encryptionkey@0"
I20260926 19:38:33.926606 28489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:33.941143 28489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:33.945281 28489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:33.945567 28489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:33.964319 28489 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45545
I20260926 19:38:33.964344 28555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45545 every 8 connection(s)
I20260926 19:38:33.965956 28489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:33.969769 28556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:33.975272 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28489
I20260926 19:38:33.975469 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:33.975921 7007 external_mini_cluster.cc:1428] Setting key 5cd88d820625ecd5f5c918686d6f4f67
I20260926 19:38:33.980988 28556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9: Bootstrap starting.
I20260926 19:38:33.984124 28556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:33.985226 28556 log.cc:824] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:33.988489 28556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9: No bootstrap required, opened a new log
I20260926 19:38:33.989370 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:33.977837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:47821 (local address 127.0.0.1:45545)
0926 19:38:33.978690 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:33.978701 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:33.978746 (+ 45us) server_negotiation.cc:407] Connection header received
0926 19:38:33.980436 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:33.980458 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:33.980917 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:33.981311 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:33.981871 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:33.983213 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:33.984584 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:33.985327 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:33.985503 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:33.986963 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:33.986970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:33.986984 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:33.987509 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:33.987574 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:33.987630 (+ 56us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:33.987774 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:33.988064 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 19:38:33.988256 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":395,"threads_started":1}
I20260926 19:38:33.993208 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:33.998299 28556 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45545 } }
I20260926 19:38:33.998822 28556 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:33.998950 28556 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfbf1a35ca594666a0e17d56ea8965c9, State: Initialized, Role: FOLLOWER
I20260926 19:38:33.999521 28556 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45545 } }
I20260926 19:38:33.999775 28556 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:33.999920 28556 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:34.000130 28556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:34.002542 28556 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45545 } }
I20260926 19:38:34.003078 28556 leader_election.cc:302] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfbf1a35ca594666a0e17d56ea8965c9; no voters:
I20260926 19:38:34.003762 28556 leader_election.cc:288] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:34.003862 28566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:34.004468 28566 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [term 1 LEADER]: Becoming Leader. State: Replica: dfbf1a35ca594666a0e17d56ea8965c9, State: Running, Role: LEADER
I20260926 19:38:34.005159 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:33.998508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:53497 (local address 127.0.0.1:45545)
0926 19:38:33.998709 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:33.998717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:33.999012 (+ 295us) server_negotiation.cc:407] Connection header received
0926 19:38:33.999233 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:33.999241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:33.999334 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:33.999453 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:34.000009 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:34.000941 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:34.002270 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:34.002759 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:34.002900 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:34.004189 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:34.004195 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:34.004199 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:34.004216 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:34.004285 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:34.004314 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:34.004455 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:34.004754 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 19:38:34.004849 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 19:38:34.005316 28566 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45545 } }
I20260926 19:38:34.007750 28566 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfbf1a35ca594666a0e17d56ea8965c9. Latest consensus state: current_term: 1 leader_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45545 } } }
I20260926 19:38:34.008172 28566 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:34.008502 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfbf1a35ca594666a0e17d56ea8965c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45545 } } }
I20260926 19:38:34.008847 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:34.012645 28556 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:34.013269 28570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:34.018657 28570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:34.028028 28570 catalog_manager.cc:1347] Generated new cluster ID: 3ce32a6e1af8492694fc85f39bdbc8eb
I20260926 19:38:34.028162 28570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:34.045037 28570 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:34.046293 28570 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:34.056442 28570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfbf1a35ca594666a0e17d56ea8965c9: Generated new TSK 0
I20260926 19:38:34.057372 28570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:34.065977 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28489
W20260926 19:38:34.067348 28551 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.167:47821
W20260926 19:38:34.068171 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28489: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:34.068339 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28489
W20260926 19:38:34.086057 28474 connection.cc:535] client connection to 127.0.0.1:45545 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45545): Connection reset by peer (error 104)
2026-09-26T19:38:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1917 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:34 dist-test-slave-phbh krb5kdc[28587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:34 dist-test-slave-phbh krb5kdc[28587](info): set up 2 sockets
Sep 26 19:38:34 dist-test-slave-phbh krb5kdc[28587](info): commencing operation
krb5kdc: starting...
W20260926 19:38:35.177317 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:35 dist-test-slave-phbh krb5kdc[28587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 19:38:35.436376 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:35.767058 28600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:35.767359 28600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:35.767540 28600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:35.774887 28600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:35.775000 28600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:35.775110 28600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:35.775214 28600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:35.775291 28600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:35.785180 28600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:35.786476 28600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:35.787746 28600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:35.799525 28608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:35.799516 28606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:35.799849 28605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:35.800354 28600 server_base.cc:1034] running on GCE node
I20260926 19:38:35.801224 28600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:35.802661 28600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:35.803889 28600 hybrid_clock.cc:648] HybridClock initialized: now 1790451515803853 us; error 45 us; skew 500 ppm
I20260926 19:38:35.804220 28600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:35 dist-test-slave-phbh krb5kdc[28587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451515, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:35.811494 28600 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 19:38:35.813062 28600 webserver.cc:460] Webserver started at http://127.0.0.1:35851/ using document root <none> and password file <none>
I20260926 19:38:35.813735 28600 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:35.813874 28600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:35.814165 28600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:35.817148 28600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "c3f00013c6d44d2f86de8b23e32ba643"
format_stamp: "Formatted at 2026-09-26 19:38:35 on dist-test-slave-phbh"
server_key: "69a886fbb6ff49897d734ecefbb6f889"
server_key_iv: "6ecf5234c95ede85d91be116b90bb7a2"
server_key_version: "encryptionkey@0"
I20260926 19:38:35.817904 28600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "c3f00013c6d44d2f86de8b23e32ba643"
format_stamp: "Formatted at 2026-09-26 19:38:35 on dist-test-slave-phbh"
server_key: "69a886fbb6ff49897d734ecefbb6f889"
server_key_iv: "6ecf5234c95ede85d91be116b90bb7a2"
server_key_version: "encryptionkey@0"
I20260926 19:38:35.822852 28600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:35.826149 28615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:35.827404 28600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 19:38:35.827585 28600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "c3f00013c6d44d2f86de8b23e32ba643"
format_stamp: "Formatted at 2026-09-26 19:38:35 on dist-test-slave-phbh"
server_key: "69a886fbb6ff49897d734ecefbb6f889"
server_key_iv: "6ecf5234c95ede85d91be116b90bb7a2"
server_key_version: "encryptionkey@0"
I20260926 19:38:35.827776 28600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:35.833516 28600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:35.837685 28600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:35.837970 28600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:35.855638 28600 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43057
I20260926 19:38:35.855686 28666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43057 every 8 connection(s)
I20260926 19:38:35.857052 28600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:35.860598 28667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:35.861543 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28600
I20260926 19:38:35.861773 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:35.862351 7007 external_mini_cluster.cc:1428] Setting key 4382acd19cd563a3575964e4d19cd2a3
I20260926 19:38:35.870164 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:35.864216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:58783 (local address 127.0.0.1:43057)
0926 19:38:35.865020 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:35.865031 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:35.865069 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:38:35.866930 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:35.866951 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:35.867290 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:35.867674 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:35.868628 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:35.868634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:35.868646 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:35.868659 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:35.868713 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:35.868731 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:35.869004 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:35.869132 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 19:38:35.869311 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":319,"threads_started":1}
I20260926 19:38:35.873827 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:35.874147 28667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643: Bootstrap starting.
I20260926 19:38:35.877135 28667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:35.878455 28667 log.cc:824] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:35.880662 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:35.878152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.167:53115 (local address 127.0.0.1:43057)
0926 19:38:35.878328 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:35.878332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:35.878800 (+ 468us) server_negotiation.cc:407] Connection header received
0926 19:38:35.878983 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:35.878988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:35.879033 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:35.879169 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:35.880049 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:35.880054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:35.880059 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:35.880076 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:35.880121 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:35.880135 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:35.880246 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:35.880359 (+ 113us) server_negotiation.cc:299] Negotiation successful
0926 19:38:35.880417 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 19:38:35.881345 28667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643: No bootstrap required, opened a new log
I20260926 19:38:35.888880 28667 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f00013c6d44d2f86de8b23e32ba643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43057 } }
I20260926 19:38:35.889245 28667 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:35.889349 28667 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f00013c6d44d2f86de8b23e32ba643, State: Initialized, Role: FOLLOWER
I20260926 19:38:35.889946 28667 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f00013c6d44d2f86de8b23e32ba643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43057 } }
I20260926 19:38:35.890197 28667 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:35.890332 28667 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:35.890504 28667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:35.892622 28667 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f00013c6d44d2f86de8b23e32ba643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43057 } }
I20260926 19:38:35.893147 28667 leader_election.cc:302] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3f00013c6d44d2f86de8b23e32ba643; no voters:
I20260926 19:38:35.893762 28667 leader_election.cc:288] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:35.893890 28677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:35.895536 28677 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [term 1 LEADER]: Becoming Leader. State: Replica: c3f00013c6d44d2f86de8b23e32ba643, State: Running, Role: LEADER
I20260926 19:38:35.896008 28667 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:35.896055 28677 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f00013c6d44d2f86de8b23e32ba643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43057 } }
I20260926 19:38:35.898972 28679 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3f00013c6d44d2f86de8b23e32ba643. Latest consensus state: current_term: 1 leader_uuid: "c3f00013c6d44d2f86de8b23e32ba643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f00013c6d44d2f86de8b23e32ba643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43057 } } }
I20260926 19:38:35.899057 28678 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3f00013c6d44d2f86de8b23e32ba643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f00013c6d44d2f86de8b23e32ba643" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43057 } } }
I20260926 19:38:35.899390 28679 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:35.899461 28678 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:35.900815 28684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:35.906989 28684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:35.915854 28684 catalog_manager.cc:1347] Generated new cluster ID: 2da3117040c74e1fb4ce23911e3ac1c0
I20260926 19:38:35.915989 28684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:35.932682 28684 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:35.933238 28684 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:35.945034 28684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3f00013c6d44d2f86de8b23e32ba643: Generated new TSK 0
I20260926 19:38:35.945883 28684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:35.952270 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28600
W20260926 19:38:35.953337 28662 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.167:58783
W20260926 19:38:35.954157 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28600: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:35.954306 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28600
2026-09-26T19:38:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1885 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:36 dist-test-slave-phbh krb5kdc[28698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:36 dist-test-slave-phbh krb5kdc[28698](info): set up 2 sockets
Sep 26 19:38:36 dist-test-slave-phbh krb5kdc[28698](info): commencing operation
krb5kdc: starting...
W20260926 19:38:37.057302 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:37 dist-test-slave-phbh krb5kdc[28698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:37.315523 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37931
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:37931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:37.649425 28711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:37.649703 28711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:37.649852 28711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:37.657054 28711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:37.657169 28711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:37.657267 28711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:37.657369 28711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:37.657426 28711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:37.667240 28711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:37931
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:37.668574 28711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:37.669867 28711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:37.681739 28716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:37.681700 28717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:37.681663 28719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:37.682519 28711 server_base.cc:1034] running on GCE node
I20260926 19:38:37.683254 28711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:37.684579 28711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:37.685787 28711 hybrid_clock.cc:648] HybridClock initialized: now 1790451517685734 us; error 46 us; skew 500 ppm
I20260926 19:38:37.686096 28711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:37 dist-test-slave-phbh krb5kdc[28698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451517, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:37.689088 28711 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:37.690528 28711 webserver.cc:460] Webserver started at http://127.6.215.254:40665/ using document root <none> and password file <none>
I20260926 19:38:37.691193 28711 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:37.691294 28711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:37.691530 28711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:37.694450 28711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "196b69870a094ec180abb3ef2f8e4e94"
format_stamp: "Formatted at 2026-09-26 19:38:37 on dist-test-slave-phbh"
server_key: "36c424ebc70fd69274cede0c5dcb2769"
server_key_iv: "2cf7bbbd984bdee3005b453fc8de6923"
server_key_version: "encryptionkey@0"
I20260926 19:38:37.695164 28711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "196b69870a094ec180abb3ef2f8e4e94"
format_stamp: "Formatted at 2026-09-26 19:38:37 on dist-test-slave-phbh"
server_key: "36c424ebc70fd69274cede0c5dcb2769"
server_key_iv: "2cf7bbbd984bdee3005b453fc8de6923"
server_key_version: "encryptionkey@0"
I20260926 19:38:37.700075 28711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:38:37.703408 28726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:37.704618 28711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260926 19:38:37.704836 28711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "196b69870a094ec180abb3ef2f8e4e94"
format_stamp: "Formatted at 2026-09-26 19:38:37 on dist-test-slave-phbh"
server_key: "36c424ebc70fd69274cede0c5dcb2769"
server_key_iv: "2cf7bbbd984bdee3005b453fc8de6923"
server_key_version: "encryptionkey@0"
I20260926 19:38:37.705024 28711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:37.716919 28711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:37.721246 28711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:37.721517 28711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:37.739223 28711 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:37931
I20260926 19:38:37.739260 28777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:37931 every 8 connection(s)
I20260926 19:38:37.740576 28711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:37.741612 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28711
I20260926 19:38:37.741789 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:37.742242 7007 external_mini_cluster.cc:1428] Setting key 1cee0ec1ed25fcb85ee4f42677e10d43
I20260926 19:38:37.744868 28778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 19:38:37 dist-test-slave-phbh krb5kdc[28698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:37.757279 28778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94: Bootstrap starting.
I20260926 19:38:37.761508 28778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:37.762849 28778 log.cc:824] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:37.766374 28778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94: No bootstrap required, opened a new log
I20260926 19:38:37.767117 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:37.744076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58203 (local address 127.6.215.254:37931)
0926 19:38:37.744866 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:37.744882 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:37.744934 (+ 52us) server_negotiation.cc:407] Connection header received
0926 19:38:37.747156 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:37.747184 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:37.747679 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:37.748117 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:37.749420 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:37.751464 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:37.752859 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:37.753901 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:37.754093 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:37.758254 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:37.758265 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:37.758282 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:37.758328 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:37.762326 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:37.763199 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:37.763208 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:37.763217 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:37.763472 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:37.764019 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:37.764023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:37.764025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:37.764692 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:37.765034 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:37.765485 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 19:38:37.765813 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":303,"threads_started":1}
I20260926 19:38:37.770999 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:37.774996 28778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196b69870a094ec180abb3ef2f8e4e94" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37931 } }
I20260926 19:38:37.775493 28778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:37.775666 28778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196b69870a094ec180abb3ef2f8e4e94, State: Initialized, Role: FOLLOWER
I20260926 19:38:37.776355 28778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196b69870a094ec180abb3ef2f8e4e94" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37931 } }
I20260926 19:38:37.776646 28778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:37.776841 28778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:37.777035 28778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:37.779816 28778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196b69870a094ec180abb3ef2f8e4e94" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37931 } }
I20260926 19:38:37.780325 28778 leader_election.cc:302] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 196b69870a094ec180abb3ef2f8e4e94; no voters:
I20260926 19:38:37.781016 28778 leader_election.cc:288] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:37.781178 28788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:37.781842 28788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [term 1 LEADER]: Becoming Leader. State: Replica: 196b69870a094ec180abb3ef2f8e4e94, State: Running, Role: LEADER
I20260926 19:38:37.782487 28788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196b69870a094ec180abb3ef2f8e4e94" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37931 } }
I20260926 19:38:37.783100 28778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:37.786181 28789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "196b69870a094ec180abb3ef2f8e4e94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196b69870a094ec180abb3ef2f8e4e94" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37931 } } }
I20260926 19:38:37.786738 28789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:37.786532 28790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 196b69870a094ec180abb3ef2f8e4e94. Latest consensus state: current_term: 1 leader_uuid: "196b69870a094ec180abb3ef2f8e4e94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "196b69870a094ec180abb3ef2f8e4e94" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37931 } } }
I20260926 19:38:37.794773 28795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:37.795109 28790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:37.795029 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:37.775404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60119 (local address 127.6.215.254:37931)
0926 19:38:37.775592 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:37.775600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:37.776122 (+ 522us) server_negotiation.cc:407] Connection header received
0926 19:38:37.776240 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:37.776251 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:37.776327 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:37.776460 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:37.778009 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:37.779168 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:37.780442 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:37.781244 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:37.781408 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:37.785445 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:37.785452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:37.785456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:37.785505 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:37.789986 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:37.791238 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:37.791247 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:37.791253 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:37.791540 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:37.792237 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:37.792245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:37.792251 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:37.792743 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:37.793173 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:37.794618 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0926 19:38:37.794801 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:38:37.801136 28795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:37.809297 28795 catalog_manager.cc:1347] Generated new cluster ID: 6315b53bd2614d5c8bf257d0506549e5
I20260926 19:38:37.809407 28795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:37.824684 28795 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:37.825968 28795 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:37.836076 28795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 196b69870a094ec180abb3ef2f8e4e94: Generated new TSK 0
I20260926 19:38:37.836927 28795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:37.843843 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28711
I20260926 19:38:37.845162 28773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:37.934901 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28711
W20260926 19:38:37.951906 28696 connection.cc:535] client connection to 127.6.215.254:37931 recv error: Network error: failed to read from TLS socket (remote: 127.6.215.254:37931): Connection reset by peer (error 104)
2026-09-26T19:38:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1980 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:38 dist-test-slave-phbh krb5kdc[28810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:38 dist-test-slave-phbh krb5kdc[28810](info): set up 2 sockets
Sep 26 19:38:38 dist-test-slave-phbh krb5kdc[28810](info): commencing operation
krb5kdc: starting...
W20260926 19:38:39.033185 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.009s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:39 dist-test-slave-phbh krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:39.293368 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:44543
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:44543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:39.627235 28823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:39.627543 28823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:39.627717 28823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:39.635166 28823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:39.635291 28823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:39.635409 28823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:39.635510 28823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:39.635584 28823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:39.645525 28823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:44543
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:44543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:39.646862 28823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:39.648164 28823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:39.659785 28829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:39.659806 28831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:39.660493 28828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:39.660895 28823 server_base.cc:1034] running on GCE node
I20260926 19:38:39.661644 28823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:39.663061 28823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:39.664275 28823 hybrid_clock.cc:648] HybridClock initialized: now 1790451519664218 us; error 55 us; skew 500 ppm
I20260926 19:38:39.664593 28823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:39 dist-test-slave-phbh krb5kdc[28810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451519, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:39.667729 28823 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:39.669210 28823 webserver.cc:460] Webserver started at http://127.6.215.254:40685/ using document root <none> and password file <none>
I20260926 19:38:39.669890 28823 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:39.670029 28823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:39.670332 28823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:39.673323 28823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "32890447f79c4b0ea4da15c2988ff37a"
format_stamp: "Formatted at 2026-09-26 19:38:39 on dist-test-slave-phbh"
server_key: "dfea1aaa28e467494fb098dd6b3e5a94"
server_key_iv: "12b1958cf6d6f6e96e9efa61b13e6f30"
server_key_version: "encryptionkey@0"
I20260926 19:38:39.674088 28823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "32890447f79c4b0ea4da15c2988ff37a"
format_stamp: "Formatted at 2026-09-26 19:38:39 on dist-test-slave-phbh"
server_key: "dfea1aaa28e467494fb098dd6b3e5a94"
server_key_iv: "12b1958cf6d6f6e96e9efa61b13e6f30"
server_key_version: "encryptionkey@0"
I20260926 19:38:39.679459 28823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 19:38:39.683207 28838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:39.684496 28823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 19:38:39.684679 28823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "32890447f79c4b0ea4da15c2988ff37a"
format_stamp: "Formatted at 2026-09-26 19:38:39 on dist-test-slave-phbh"
server_key: "dfea1aaa28e467494fb098dd6b3e5a94"
server_key_iv: "12b1958cf6d6f6e96e9efa61b13e6f30"
server_key_version: "encryptionkey@0"
I20260926 19:38:39.684929 28823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:39.703593 28823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:39.707986 28823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:39.708290 28823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:39.726689 28823 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:44543
I20260926 19:38:39.726702 28889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:44543 every 8 connection(s)
I20260926 19:38:39.728112 28823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:39.731765 28890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:39.737735 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28823
I20260926 19:38:39.737938 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:39.738363 7007 external_mini_cluster.cc:1428] Setting key f5c0308002ce4d63659ab2f7411470be
I20260926 19:38:39.742974 28890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a: Bootstrap starting.
I20260926 19:38:39.746999 28890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:39.748018 28890 log.cc:824] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:39.751286 28890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a: No bootstrap required, opened a new log
I20260926 19:38:39.753412 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:39.740223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33025 (local address 127.6.215.254:44543)
0926 19:38:39.741042 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:39.741059 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:39.741107 (+ 48us) server_negotiation.cc:407] Connection header received
0926 19:38:39.743490 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:39.743521 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:39.744021 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:39.744466 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:39.745056 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:39.747050 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:39.748454 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:39.749327 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:39.749508 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:39.750666 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:39.750679 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:39.750698 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:39.751409 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:39.751499 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:39.751531 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:39.751688 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:39.751949 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 19:38:39.752119 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":323,"threads_started":1}
I20260926 19:38:39.757947 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:39.760375 28890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32890447f79c4b0ea4da15c2988ff37a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44543 } }
I20260926 19:38:39.760883 28890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:39.760975 28890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32890447f79c4b0ea4da15c2988ff37a, State: Initialized, Role: FOLLOWER
I20260926 19:38:39.761559 28890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32890447f79c4b0ea4da15c2988ff37a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44543 } }
I20260926 19:38:39.761804 28890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:39.761942 28890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:39.762112 28890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:39.764253 28890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32890447f79c4b0ea4da15c2988ff37a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44543 } }
I20260926 19:38:39.764802 28890 leader_election.cc:302] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32890447f79c4b0ea4da15c2988ff37a; no voters:
I20260926 19:38:39.765573 28890 leader_election.cc:288] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:39.765756 28898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:39.766513 28898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [term 1 LEADER]: Becoming Leader. State: Replica: 32890447f79c4b0ea4da15c2988ff37a, State: Running, Role: LEADER
I20260926 19:38:39.767279 28898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32890447f79c4b0ea4da15c2988ff37a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44543 } }
I20260926 19:38:39.768944 28890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:39.770776 28900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32890447f79c4b0ea4da15c2988ff37a. Latest consensus state: current_term: 1 leader_uuid: "32890447f79c4b0ea4da15c2988ff37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32890447f79c4b0ea4da15c2988ff37a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44543 } } }
I20260926 19:38:39.770893 28901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32890447f79c4b0ea4da15c2988ff37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32890447f79c4b0ea4da15c2988ff37a" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 44543 } } }
I20260926 19:38:39.771214 28900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:39.771287 28901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:39.772202 28905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:39.777187 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:39.766818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57737 (local address 127.6.215.254:44543)
0926 19:38:39.767037 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:39.767045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:39.768322 (+ 1277us) server_negotiation.cc:407] Connection header received
0926 19:38:39.768566 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:39.768570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:39.768634 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:39.768741 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:39.769339 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:39.770733 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:39.772911 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:39.773641 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:39.773801 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:39.775769 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:39.775780 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:39.775787 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:39.775818 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:39.775879 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:39.775908 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:39.776037 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:39.776593 (+ 556us) server_negotiation.cc:299] Negotiation successful
0926 19:38:39.776670 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 19:38:39.782614 28905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:39.791288 28905 catalog_manager.cc:1347] Generated new cluster ID: a89cc86a2b024ba6b7b3f0cef1e63546
I20260926 19:38:39.791410 28905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:39.820920 28905 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:39.822793 28905 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:39.845503 28905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32890447f79c4b0ea4da15c2988ff37a: Generated new TSK 0
I20260926 19:38:39.846431 28905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:39.860909 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28823
W20260926 19:38:39.862064 28885 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33025
W20260926 19:38:39.862774 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28823: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:39.862926 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28823
W20260926 19:38:39.879899 28808 connection.cc:535] client connection to 127.6.215.254:44543 recv error: Network error: failed to read from TLS socket (remote: 127.6.215.254:44543): Connection reset by peer (error 104)
2026-09-26T19:38:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:39 dist-test-slave-phbh krb5kdc[28921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:39 dist-test-slave-phbh krb5kdc[28921](info): set up 2 sockets
Sep 26 19:38:39 dist-test-slave-phbh krb5kdc[28921](info): commencing operation
krb5kdc: starting...
W20260926 19:38:40.989280 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:41 dist-test-slave-phbh krb5kdc[28921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:41.249936 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:36163
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:36163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:41.582173 28934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:41.582522 28934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:41.582700 28934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:41.590034 28934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:41.590204 28934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:41.590299 28934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:41.590409 28934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:41.590485 28934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:41.601421 28934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:36163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:36163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:41.602823 28934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:41.604101 28934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:41.615839 28939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:41.615896 28940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:41.615919 28942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:41.617201 28934 server_base.cc:1034] running on GCE node
I20260926 19:38:41.617911 28934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:41.619199 28934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:41.620410 28934 hybrid_clock.cc:648] HybridClock initialized: now 1790451521620369 us; error 37 us; skew 500 ppm
I20260926 19:38:41.620748 28934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:41 dist-test-slave-phbh krb5kdc[28921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451521, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:41.638672 28934 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:41.640262 28934 webserver.cc:460] Webserver started at http://127.6.215.254:35115/ using document root <none> and password file <none>
I20260926 19:38:41.641001 28934 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:41.641111 28934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:41.641377 28934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:41.644471 28934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff"
format_stamp: "Formatted at 2026-09-26 19:38:41 on dist-test-slave-phbh"
server_key: "93a140f96fb997125cc0cdf98bb233d9"
server_key_iv: "823e3118ec1d08291b3c16c3d3fee5db"
server_key_version: "encryptionkey@0"
I20260926 19:38:41.645280 28934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff"
format_stamp: "Formatted at 2026-09-26 19:38:41 on dist-test-slave-phbh"
server_key: "93a140f96fb997125cc0cdf98bb233d9"
server_key_iv: "823e3118ec1d08291b3c16c3d3fee5db"
server_key_version: "encryptionkey@0"
I20260926 19:38:41.650419 28934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:38:41.653786 28949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:41.655138 28934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:38:41.655320 28934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff"
format_stamp: "Formatted at 2026-09-26 19:38:41 on dist-test-slave-phbh"
server_key: "93a140f96fb997125cc0cdf98bb233d9"
server_key_iv: "823e3118ec1d08291b3c16c3d3fee5db"
server_key_version: "encryptionkey@0"
I20260926 19:38:41.655505 28934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:41.661180 28934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:41.664983 28934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:41.665261 28934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:41.683290 28934 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:36163
I20260926 19:38:41.683348 29000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:36163 every 8 connection(s)
I20260926 19:38:41.684758 28934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:41.688627 29001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:41.691344 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 28934
I20260926 19:38:41.691525 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:41.692032 7007 external_mini_cluster.cc:1428] Setting key b98b6ad34593bd3876eae7d3a19819f3
I20260926 19:38:41.699724 29004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:41.693920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48463 (local address 127.6.215.254:36163)
0926 19:38:41.694725 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:41.694736 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:41.694777 (+ 41us) server_negotiation.cc:407] Connection header received
0926 19:38:41.696402 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:41.696425 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:41.696761 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:41.697167 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:41.698200 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:41.698206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:41.698219 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:41.698236 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:41.698298 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:41.698324 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:41.698493 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:41.698590 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 19:38:41.698789 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":388,"threads_started":1}
I20260926 19:38:41.700945 29001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff: Bootstrap starting.
I20260926 19:38:41.703341 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:41.704321 29001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:41.705580 29001 log.cc:824] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:41.708308 29001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff: No bootstrap required, opened a new log
I20260926 19:38:41.711267 29004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:41.708568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51437 (local address 127.6.215.254:36163)
0926 19:38:41.708813 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:41.708822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:41.709256 (+ 434us) server_negotiation.cc:407] Connection header received
0926 19:38:41.709437 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:41.709441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:41.709494 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:41.709574 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:41.710578 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:41.710583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:41.710586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:41.710604 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:41.710646 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:41.710662 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:41.710776 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:41.710899 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 19:38:41.710949 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 19:38:41.716648 29001 raft_consensus.cc:348] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36163 } }
I20260926 19:38:41.717069 29001 raft_consensus.cc:374] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:41.717171 29001 raft_consensus.cc:729] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f367699a8fcf4cd4adf8b6ef5ae0b1ff, State: Initialized, Role: FOLLOWER
I20260926 19:38:41.717715 29001 consensus_queue.cc:260] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36163 } }
I20260926 19:38:41.717960 29001 raft_consensus.cc:388] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:41.718102 29001 raft_consensus.cc:482] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:41.718273 29001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:41.720398 29001 raft_consensus.cc:504] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36163 } }
I20260926 19:38:41.720937 29001 leader_election.cc:302] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f367699a8fcf4cd4adf8b6ef5ae0b1ff; no voters:
I20260926 19:38:41.721555 29001 leader_election.cc:288] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:41.721683 29011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:41.722901 29011 raft_consensus.cc:686] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [term 1 LEADER]: Becoming Leader. State: Replica: f367699a8fcf4cd4adf8b6ef5ae0b1ff, State: Running, Role: LEADER
I20260926 19:38:41.723409 29011 consensus_queue.cc:237] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36163 } }
I20260926 19:38:41.723730 29001 sys_catalog.cc:564] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:41.726461 29013 sys_catalog.cc:455] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader f367699a8fcf4cd4adf8b6ef5ae0b1ff. Latest consensus state: current_term: 1 leader_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36163 } } }
I20260926 19:38:41.726553 29012 sys_catalog.cc:455] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f367699a8fcf4cd4adf8b6ef5ae0b1ff" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 36163 } } }
I20260926 19:38:41.726853 29013 sys_catalog.cc:458] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:41.726933 29012 sys_catalog.cc:458] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:41.729048 29018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:41.734265 29018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:41.743042 29018 catalog_manager.cc:1347] Generated new cluster ID: 0484a0cc09d04006b7914a5c769eeac3
I20260926 19:38:41.743158 29018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:41.754204 29018 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:41.754662 29018 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:41.768314 29018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f367699a8fcf4cd4adf8b6ef5ae0b1ff: Generated new TSK 0
I20260926 19:38:41.769393 29018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:41.774210 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28934
W20260926 19:38:41.775552 28996 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48463
W20260926 19:38:41.776314 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 28934: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:41.776460 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 28934
2026-09-26T19:38:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1913 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:41 dist-test-slave-phbh krb5kdc[29032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:41 dist-test-slave-phbh krb5kdc[29032](info): set up 2 sockets
Sep 26 19:38:41 dist-test-slave-phbh krb5kdc[29032](info): commencing operation
krb5kdc: starting...
W20260926 19:38:42.889454 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:43 dist-test-slave-phbh krb5kdc[29032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:43.148118 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46589
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:39447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:46589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:43.478298 29045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:43.478595 29045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:43.478775 29045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:43.485996 29045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:43.486143 29045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:43.486235 29045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:43.486326 29045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:43.486382 29045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:43.496250 29045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:39447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:46589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:46589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:43.497567 29045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:43.498811 29045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:43.510380 29050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:43.510460 29053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:43.510499 29051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:43.511276 29045 server_base.cc:1034] running on GCE node
I20260926 19:38:43.512075 29045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:43.513532 29045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:43.514745 29045 hybrid_clock.cc:648] HybridClock initialized: now 1790451523514700 us; error 34 us; skew 500 ppm
I20260926 19:38:43.515098 29045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:43 dist-test-slave-phbh krb5kdc[29032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:43.518232 29045 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:43.519800 29045 webserver.cc:460] Webserver started at http://127.6.215.254:46049/ using document root <none> and password file <none>
I20260926 19:38:43.520486 29045 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:43.520654 29045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:43.521025 29045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:43.524081 29045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "dbbd308452dd4936a6fdb1546c006a15"
format_stamp: "Formatted at 2026-09-26 19:38:43 on dist-test-slave-phbh"
server_key: "0a3961d929e23625f0792df0a1b8d8d1"
server_key_iv: "a0a49a80b1fe2b6702a415cef6d47c71"
server_key_version: "encryptionkey@0"
I20260926 19:38:43.524924 29045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "dbbd308452dd4936a6fdb1546c006a15"
format_stamp: "Formatted at 2026-09-26 19:38:43 on dist-test-slave-phbh"
server_key: "0a3961d929e23625f0792df0a1b8d8d1"
server_key_iv: "a0a49a80b1fe2b6702a415cef6d47c71"
server_key_version: "encryptionkey@0"
I20260926 19:38:43.530138 29045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 19:38:43.533828 29060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:43.535162 29045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 19:38:43.535367 29045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "dbbd308452dd4936a6fdb1546c006a15"
format_stamp: "Formatted at 2026-09-26 19:38:43 on dist-test-slave-phbh"
server_key: "0a3961d929e23625f0792df0a1b8d8d1"
server_key_iv: "a0a49a80b1fe2b6702a415cef6d47c71"
server_key_version: "encryptionkey@0"
I20260926 19:38:43.535578 29045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:43.549438 29045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:43.553550 29045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:43.553845 29045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:43.575454 29045 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:46589
I20260926 19:38:43.575474 29111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:46589 every 8 connection(s)
I20260926 19:38:43.576900 29045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:43.581188 29112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:43.582788 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29045
I20260926 19:38:43.582978 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:43.583601 7007 external_mini_cluster.cc:1428] Setting key 20134bf303c81c0fda5307da8b92f2fb
I20260926 19:38:43.599478 29112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15: Bootstrap starting.
I20260926 19:38:43.602680 29112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:43.603744 29112 log.cc:824] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:43.606621 29112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15: No bootstrap required, opened a new log
Sep 26 19:38:43 dist-test-slave-phbh krb5kdc[29032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:43.614612 29112 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbbd308452dd4936a6fdb1546c006a15" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46589 } }
I20260926 19:38:43.615041 29112 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:43.615154 29112 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbbd308452dd4936a6fdb1546c006a15, State: Initialized, Role: FOLLOWER
I20260926 19:38:43.615825 29112 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbbd308452dd4936a6fdb1546c006a15" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46589 } }
I20260926 19:38:43.616094 29112 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:43.616253 29112 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:43.616492 29112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:43.618654 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:43.585878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51975 (local address 127.6.215.254:46589)
0926 19:38:43.597419 (+ 11541us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:43.597433 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:43.598142 (+ 709us) server_negotiation.cc:407] Connection header received
0926 19:38:43.599902 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:43.599926 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:43.600349 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:43.600734 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:43.602257 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:43.603688 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:43.605448 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:43.606131 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:43.606335 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:43.610454 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:43.610465 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:43.610480 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:43.610529 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:43.613849 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:43.614567 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:43.614577 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:43.614584 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:43.614773 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:43.615250 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:43.615254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:43.615256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:43.615811 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:43.616146 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:43.616630 (+ 484us) server_negotiation.cc:299] Negotiation successful
0926 19:38:43.617126 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11287,"thread_start_us":11031,"threads_started":1}
I20260926 19:38:43.619616 29112 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbbd308452dd4936a6fdb1546c006a15" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46589 } }
I20260926 19:38:43.620148 29112 leader_election.cc:302] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbbd308452dd4936a6fdb1546c006a15; no voters:
I20260926 19:38:43.620832 29112 leader_election.cc:288] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:43.620934 29117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:43.621649 29117 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [term 1 LEADER]: Becoming Leader. State: Replica: dbbd308452dd4936a6fdb1546c006a15, State: Running, Role: LEADER
I20260926 19:38:43.622388 29117 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbbd308452dd4936a6fdb1546c006a15" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46589 } }
I20260926 19:38:43.622967 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:43.623667 29112 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:43.625872 29119 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbbd308452dd4936a6fdb1546c006a15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbbd308452dd4936a6fdb1546c006a15" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46589 } } }
I20260926 19:38:43.626319 29119 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:43.629292 29118 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbbd308452dd4936a6fdb1546c006a15. Latest consensus state: current_term: 1 leader_uuid: "dbbd308452dd4936a6fdb1546c006a15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbbd308452dd4936a6fdb1546c006a15" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 46589 } } }
I20260926 19:38:43.629739 29118 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:43.629802 29123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:43.636824 29123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:43.650470 29123 catalog_manager.cc:1347] Generated new cluster ID: d9e1d715397e4445a540525008a25992
I20260926 19:38:43.650620 29123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:43.672941 29123 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:43.674839 29123 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:43.683864 29123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbbd308452dd4936a6fdb1546c006a15: Generated new TSK 0
I20260926 19:38:43.684985 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:43.656703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40153 (local address 127.6.215.254:46589)
0926 19:38:43.656922 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:43.656928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:43.664016 (+ 7088us) server_negotiation.cc:407] Connection header received
0926 19:38:43.664516 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:43.664522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:43.664616 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:43.664704 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:43.666916 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:43.668411 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:43.670933 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:43.671940 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:43.672073 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:43.676262 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:43.676272 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:43.676280 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:43.676334 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:43.681702 (+ 5368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:43.682437 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:43.682442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:43.682446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:43.682605 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:43.683007 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:43.683012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:43.683015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:43.683467 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:43.683724 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:43.684327 (+ 603us) server_negotiation.cc:299] Negotiation successful
0926 19:38:43.684578 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 19:38:43.685055 29123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:43.696760 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29045
I20260926 19:38:43.697952 29107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:43.787375 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29045
2026-09-26T19:38:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2011 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:43 dist-test-slave-phbh krb5kdc[29144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:43 dist-test-slave-phbh krb5kdc[29144](info): set up 2 sockets
Sep 26 19:38:43 dist-test-slave-phbh krb5kdc[29144](info): commencing operation
krb5kdc: starting...
W20260926 19:38:44.921324 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:45 dist-test-slave-phbh krb5kdc[29144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:45.180495 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42383
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:42383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:45.512225 29157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:45.512599 29157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:45.512758 29157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:45.520001 29157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:45.520161 29157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:45.520263 29157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:45.520370 29157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:45.520443 29157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:45.530685 29157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:42383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:42383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:45.532029 29157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:45.533387 29157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:45.545295 29162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:45.545167 29163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:45.545496 29165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:45.546176 29157 server_base.cc:1034] running on GCE node
I20260926 19:38:45.546986 29157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:45.548432 29157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:45.549683 29157 hybrid_clock.cc:648] HybridClock initialized: now 1790451525549595 us; error 65 us; skew 500 ppm
I20260926 19:38:45.550051 29157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:45 dist-test-slave-phbh krb5kdc[29144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451525, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:45.553176 29157 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:45.554795 29157 webserver.cc:460] Webserver started at http://127.6.215.254:32969/ using document root <none> and password file <none>
I20260926 19:38:45.555500 29157 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:45.555647 29157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:45.555948 29157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:45.558959 29157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "d0c5e6baccbb432895ec59508ad70c74"
format_stamp: "Formatted at 2026-09-26 19:38:45 on dist-test-slave-phbh"
server_key: "f577fb42c664995b8b2db17627c66b05"
server_key_iv: "51b9ace8c14fbcd1c534d7213cc02ca7"
server_key_version: "encryptionkey@0"
I20260926 19:38:45.559715 29157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "d0c5e6baccbb432895ec59508ad70c74"
format_stamp: "Formatted at 2026-09-26 19:38:45 on dist-test-slave-phbh"
server_key: "f577fb42c664995b8b2db17627c66b05"
server_key_iv: "51b9ace8c14fbcd1c534d7213cc02ca7"
server_key_version: "encryptionkey@0"
I20260926 19:38:45.564819 29157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 19:38:45.568169 29172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:45.569521 29157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:38:45.569702 29157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "d0c5e6baccbb432895ec59508ad70c74"
format_stamp: "Formatted at 2026-09-26 19:38:45 on dist-test-slave-phbh"
server_key: "f577fb42c664995b8b2db17627c66b05"
server_key_iv: "51b9ace8c14fbcd1c534d7213cc02ca7"
server_key_version: "encryptionkey@0"
I20260926 19:38:45.569897 29157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:45.592324 29157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:45.596729 29157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:45.597086 29157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:45.615759 29157 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:42383
I20260926 19:38:45.615854 29223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:42383 every 8 connection(s)
I20260926 19:38:45.617166 29157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:45.617640 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29157
I20260926 19:38:45.617837 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:45.618474 7007 external_mini_cluster.cc:1428] Setting key df5dd168ec4eb371a1079b5c0dec412f
I20260926 19:38:45.621421 29224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:45.632464 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:45.620475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45407 (local address 127.6.215.254:42383)
0926 19:38:45.621739 (+ 1264us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:45.621756 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:45.621801 (+ 45us) server_negotiation.cc:407] Connection header received
0926 19:38:45.623433 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:45.623453 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:45.623849 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:45.624165 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:45.624740 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:45.626229 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:45.628087 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:45.628726 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:45.628970 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:45.630238 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:45.630249 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:45.630263 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:45.630820 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:45.630898 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:45.630922 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:45.631061 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:45.631303 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 19:38:45.631564 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":769,"threads_started":1}
I20260926 19:38:45.633450 29224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74: Bootstrap starting.
I20260926 19:38:45.636212 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:45.637049 29224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:45.638003 29224 log.cc:824] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:45.641031 29224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74: No bootstrap required, opened a new log
I20260926 19:38:45.647341 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:45.640690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46731 (local address 127.6.215.254:42383)
0926 19:38:45.640892 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:45.640897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:45.641358 (+ 461us) server_negotiation.cc:407] Connection header received
0926 19:38:45.641518 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:45.641528 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:45.641616 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:45.641728 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:45.642184 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:45.643320 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:45.644637 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:45.645347 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:45.645460 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:45.646571 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:45.646579 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:45.646588 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:45.646607 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:45.646651 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:45.646670 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:45.646765 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:45.646899 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 19:38:45.646999 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:38:45.650493 29224 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c5e6baccbb432895ec59508ad70c74" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42383 } }
I20260926 19:38:45.650844 29224 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:45.650920 29224 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c5e6baccbb432895ec59508ad70c74, State: Initialized, Role: FOLLOWER
I20260926 19:38:45.651485 29224 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c5e6baccbb432895ec59508ad70c74" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42383 } }
I20260926 19:38:45.651765 29224 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:45.651923 29224 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:45.652122 29224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:45.654340 29224 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c5e6baccbb432895ec59508ad70c74" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42383 } }
I20260926 19:38:45.654827 29224 leader_election.cc:302] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c5e6baccbb432895ec59508ad70c74; no voters:
I20260926 19:38:45.655366 29224 leader_election.cc:288] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:45.655552 29234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:45.656980 29234 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [term 1 LEADER]: Becoming Leader. State: Replica: d0c5e6baccbb432895ec59508ad70c74, State: Running, Role: LEADER
I20260926 19:38:45.657493 29234 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c5e6baccbb432895ec59508ad70c74" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42383 } }
I20260926 19:38:45.657740 29224 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:45.660158 29236 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c5e6baccbb432895ec59508ad70c74. Latest consensus state: current_term: 1 leader_uuid: "d0c5e6baccbb432895ec59508ad70c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c5e6baccbb432895ec59508ad70c74" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42383 } } }
I20260926 19:38:45.660189 29235 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0c5e6baccbb432895ec59508ad70c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c5e6baccbb432895ec59508ad70c74" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 42383 } } }
I20260926 19:38:45.660615 29235 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:45.660615 29236 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:45.661804 29242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:45.668306 29242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:45.677696 29242 catalog_manager.cc:1347] Generated new cluster ID: 5134745a598c4f0690901f2a2b9060f2
I20260926 19:38:45.677835 29242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:45.700376 29242 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:45.701773 29242 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:45.713207 29242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c5e6baccbb432895ec59508ad70c74: Generated new TSK 0
I20260926 19:38:45.714010 29242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:45.725284 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29157
W20260926 19:38:45.726351 29219 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45407
W20260926 19:38:45.727033 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29157: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:45.727182 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29157
2026-09-26T19:38:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1939 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:45 dist-test-slave-phbh krb5kdc[29255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:45 dist-test-slave-phbh krb5kdc[29255](info): set up 2 sockets
Sep 26 19:38:45 dist-test-slave-phbh krb5kdc[29255](info): commencing operation
krb5kdc: starting...
W20260926 19:38:46.825215 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:46 dist-test-slave-phbh krb5kdc[29255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:47.080080 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37269
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:37269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:47.415700 29268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:47.416030 29268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:47.416218 29268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:47.423883 29268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:47.424001 29268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:47.424047 29268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:47.424109 29268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:47.424153 29268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:47.434630 29268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:37269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:37269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:47.435886 29268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:47.437143 29268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:47.449064 29273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:47.449008 29274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:47.449038 29276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:47.449836 29268 server_base.cc:1034] running on GCE node
I20260926 19:38:47.450616 29268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:47.452033 29268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:47.453269 29268 hybrid_clock.cc:648] HybridClock initialized: now 1790451527453202 us; error 53 us; skew 500 ppm
I20260926 19:38:47.453655 29268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:47 dist-test-slave-phbh krb5kdc[29255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451527, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:47.463950 29268 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:47.465750 29268 webserver.cc:460] Webserver started at http://127.6.215.254:39307/ using document root <none> and password file <none>
I20260926 19:38:47.466497 29268 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:47.466652 29268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:47.466984 29268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:47.470101 29268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e"
format_stamp: "Formatted at 2026-09-26 19:38:47 on dist-test-slave-phbh"
server_key: "fd375358ac63bb8c1ddd5a84eb63ff2c"
server_key_iv: "e41e512284a79c00f0e97d7fd1a0e71b"
server_key_version: "encryptionkey@0"
I20260926 19:38:47.470911 29268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e"
format_stamp: "Formatted at 2026-09-26 19:38:47 on dist-test-slave-phbh"
server_key: "fd375358ac63bb8c1ddd5a84eb63ff2c"
server_key_iv: "e41e512284a79c00f0e97d7fd1a0e71b"
server_key_version: "encryptionkey@0"
I20260926 19:38:47.476285 29268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 19:38:47.479892 29283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:47.481351 29268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:47.481539 29268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e"
format_stamp: "Formatted at 2026-09-26 19:38:47 on dist-test-slave-phbh"
server_key: "fd375358ac63bb8c1ddd5a84eb63ff2c"
server_key_iv: "e41e512284a79c00f0e97d7fd1a0e71b"
server_key_version: "encryptionkey@0"
I20260926 19:38:47.481745 29268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:47.487954 29268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:47.492344 29268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:47.492662 29268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:47.511854 29268 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:37269
I20260926 19:38:47.511885 29334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:37269 every 8 connection(s)
I20260926 19:38:47.513307 29268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:47.515476 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29268
I20260926 19:38:47.515691 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:47.516117 7007 external_mini_cluster.cc:1428] Setting key d71d7972864991a637f770aec149d506
I20260926 19:38:47.517865 29335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:47.524044 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:47.518009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33239 (local address 127.6.215.254:37269)
0926 19:38:47.518829 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:47.518841 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:47.518879 (+ 38us) server_negotiation.cc:407] Connection header received
0926 19:38:47.520500 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:47.520522 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:47.521012 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:47.521368 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:47.522477 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:47.522483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:47.522495 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:47.522509 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:47.522582 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:47.522601 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:47.522763 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:47.522893 (+ 130us) server_negotiation.cc:299] Negotiation successful
0926 19:38:47.523099 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":379,"threads_started":1}
I20260926 19:38:47.527685 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:47.531368 29335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e: Bootstrap starting.
I20260926 19:38:47.535544 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:47.533057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32875 (local address 127.6.215.254:37269)
0926 19:38:47.533274 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:47.533280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:47.533625 (+ 345us) server_negotiation.cc:407] Connection header received
0926 19:38:47.533834 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:47.533839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:47.533888 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:47.533962 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:47.534921 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:47.534925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:47.534928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 19:38:47.534942 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:47.534978 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 19:38:47.534992 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:47.535076 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:47.535210 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 19:38:47.535255 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:38:47.535550 29335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:47.536890 29335 log.cc:824] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:47.539901 29335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e: No bootstrap required, opened a new log
I20260926 19:38:47.547420 29335 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37269 } }
I20260926 19:38:47.547775 29335 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 19:38:47.547842 29335 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b0b2e2f7e945f1a1ad1c53a68cd84e, State: Initialized, Role: FOLLOWER
I20260926 19:38:47.548334 29335 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37269 } }
I20260926 19:38:47.548523 29335 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 19:38:47.548619 29335 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 19:38:47.548743 29335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:47.551236 29335 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37269 } }
I20260926 19:38:47.551774 29335 leader_election.cc:302] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0b0b2e2f7e945f1a1ad1c53a68cd84e; no voters:
I20260926 19:38:47.552292 29335 leader_election.cc:288] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 19:38:47.552440 29345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:47.553164 29345 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [term 1 LEADER]: Becoming Leader. State: Replica: c0b0b2e2f7e945f1a1ad1c53a68cd84e, State: Running, Role: LEADER
I20260926 19:38:47.553727 29345 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37269 } }
I20260926 19:38:47.554445 29335 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:47.556826 29347 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0b0b2e2f7e945f1a1ad1c53a68cd84e. Latest consensus state: current_term: 1 leader_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37269 } } }
I20260926 19:38:47.557231 29347 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:47.556818 29346 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b0b2e2f7e945f1a1ad1c53a68cd84e" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 37269 } } }
I20260926 19:38:47.561933 29346 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:47.561986 29351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:47.567782 29351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:47.576885 29351 catalog_manager.cc:1347] Generated new cluster ID: 9106522fac9b43c28a2b20e70fdf7672
I20260926 19:38:47.577015 29351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:47.599012 29351 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:47.599483 29351 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:47.606459 29351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0b0b2e2f7e945f1a1ad1c53a68cd84e: Generated new TSK 0
I20260926 19:38:47.607264 29351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:47.616663 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29268
W20260926 19:38:47.618579 29330 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33239
W20260926 19:38:47.619318 7007 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29268: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 19:38:47.619495 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29268
2026-09-26T19:38:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1891 ms)
[----------] 18 tests from AuthTokenIssuingTest (35205 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:47 dist-test-slave-phbh krb5kdc[29366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:47 dist-test-slave-phbh krb5kdc[29366](info): set up 2 sockets
Sep 26 19:38:47 dist-test-slave-phbh krb5kdc[29366](info): commencing operation
krb5kdc: starting...
W20260926 19:38:48.729489 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:48 dist-test-slave-phbh krb5kdc[29366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:48.998412 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:41137
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:34321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:41137,127.6.215.253:33905,127.6.215.252:38627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:49.331367 29379 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 19:38:49.331740 29379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:49.331871 29379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:49.332026 29379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:49.339520 29379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:49.339615 29379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:49.339720 29379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:49.339797 29379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:49.339887 29379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:49.349800 29379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:41137,127.6.215.253:33905,127.6.215.252:38627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:41137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:49.351416 29379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:49.352869 29379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:49.364755 29384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:49.364939 29387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:49.364755 29385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:49.365590 29379 server_base.cc:1034] running on GCE node
I20260926 19:38:49.366394 29379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:49.367805 29379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:49.369041 29379 hybrid_clock.cc:648] HybridClock initialized: now 1790451529368968 us; error 51 us; skew 500 ppm
I20260926 19:38:49.369374 29379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:49 dist-test-slave-phbh krb5kdc[29366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:49.372538 29379 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:49.374082 29379 webserver.cc:460] Webserver started at http://127.6.215.254:45999/ using document root <none> and password file <none>
I20260926 19:38:49.374759 29379 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:49.374898 29379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:49.375211 29379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:49.378648 29379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "83ef5f8d6b2a4b8094b9b033618c04b7"
format_stamp: "Formatted at 2026-09-26 19:38:49 on dist-test-slave-phbh"
server_key: "5b1ef149f838ea15a31983b8af1d04fb"
server_key_iv: "33e18bc73a82ebdab04c04580bb80438"
server_key_version: "encryptionkey@0"
I20260926 19:38:49.379434 29379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "83ef5f8d6b2a4b8094b9b033618c04b7"
format_stamp: "Formatted at 2026-09-26 19:38:49 on dist-test-slave-phbh"
server_key: "5b1ef149f838ea15a31983b8af1d04fb"
server_key_iv: "33e18bc73a82ebdab04c04580bb80438"
server_key_version: "encryptionkey@0"
I20260926 19:38:49.384611 29379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 19:38:49.388060 29394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:49.389516 29379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:49.389693 29379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "83ef5f8d6b2a4b8094b9b033618c04b7"
format_stamp: "Formatted at 2026-09-26 19:38:49 on dist-test-slave-phbh"
server_key: "5b1ef149f838ea15a31983b8af1d04fb"
server_key_iv: "33e18bc73a82ebdab04c04580bb80438"
server_key_version: "encryptionkey@0"
I20260926 19:38:49.389925 29379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:49.408756 29379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:49.414655 29379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:49.414983 29379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:49.434667 29379 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:41137
I20260926 19:38:49.434671 29445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:41137 every 8 connection(s)
I20260926 19:38:49.436233 29379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:49.439976 29446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:49.443495 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29379
I20260926 19:38:49.443717 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:49.444352 7007 external_mini_cluster.cc:1428] Setting key 7134db63d212c03f8933a99285372ed1
I20260926 19:38:49.446842 29446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:49 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.253@KRBTEST.COM" created.
I20260926 19:38:49.470423 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.450179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.6.215.254:41137)
0926 19:38:49.451029 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:49.451049 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:49.451520 (+ 471us) server_negotiation.cc:407] Connection header received
0926 19:38:49.453633 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:49.453639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:49.453952 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:49.454161 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:49.455477 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.456672 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.458082 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.458667 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.458881 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:49.462820 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:49.462828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:49.462841 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:49.462870 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:49.465838 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:49.466818 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:49.466826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:49.466828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:49.467014 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:49.467507 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:49.467511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:49.467513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:49.468147 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:49.468354 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:49.469004 (+ 650us) server_negotiation.cc:299] Negotiation successful
0926 19:38:49.469378 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":212,"server-negotiator.queue_time_us":596,"thread_start_us":374,"threads_started":1}
I20260926 19:38:49.470453 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.450189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41137 (local address 127.0.0.1:37020)
0926 19:38:49.451352 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.451396 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:49.453249 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:49.454189 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:49.454195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:49.454898 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:49.455264 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:49.455281 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.456873 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.456878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:49.457941 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:49.457953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.458924 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.458931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:49.459221 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:49.459915 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:49.459939 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:49.462539 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:49.465992 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:49.466009 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:49.466023 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:49.466623 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:49.467158 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:49.467166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:49.467172 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:49.467386 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:49.468309 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:49.468318 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:49.468524 (+ 206us) client_negotiation.cc:769] Sending connection context
0926 19:38:49.468837 (+ 313us) client_negotiation.cc:240] Negotiation successful
0926 19:38:49.469249 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":41,"thread_start_us":713,"threads_started":1}
I20260926 19:38:49.473766 29446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:38:49.474920 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.474522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50516)
0926 19:38:49.474693 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.474807 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 19:38:49.475158 29395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
I20260926 19:38:49.476854 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.476420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50526)
0926 19:38:49.476578 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.476646 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 19:38:49.477068 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:33905: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab.
Entry for principal kudu/127.6.215.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab.
I20260926 19:38:49.504474 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:33905
--webserver_interface=127.6.215.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab
--principal=kudu/127.6.215.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:34321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:41137,127.6.215.253:33905,127.6.215.252:38627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:38:49.526386 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } attempt: 1
I20260926 19:38:49.527652 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.527180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50534)
0926 19:38:49.527403 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.527521 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260926 19:38:49.528731 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.528378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50546)
0926 19:38:49.528522 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.528590 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 19:38:49.529008 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:33905: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
I20260926 19:38:49.597306 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } attempt: 2
I20260926 19:38:49.598603 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.598105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50560)
0926 19:38:49.598366 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.598462 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 19:38:49.599682 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.599309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50564)
0926 19:38:49.599484 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.599553 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 19:38:49.599876 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:33905: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
I20260926 19:38:49.691172 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } attempt: 3
I20260926 19:38:49.692480 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.691971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50570)
0926 19:38:49.692202 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.692320 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260926 19:38:49.693593 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.693216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50582)
0926 19:38:49.693385 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.693452 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 19:38:49.693771 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:33905: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
I20260926 19:38:49.837129 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } attempt: 4
I20260926 19:38:49.838321 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.837799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50590)
0926 19:38:49.838101 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.838184 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260926 19:38:49.838181 29453 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 19:38:49.838618 29453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:49.838758 29453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:49.838900 29453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 19:38:49.839445 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.839086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50600)
0926 19:38:49.839244 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.839325 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 19:38:49.839685 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:33905: Network error: Client connection negotiation failed: client connection to 127.6.215.253:33905: connect: Connection refused (error 111)
W20260926 19:38:49.846556 29453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:49.846685 29453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:49.846807 29453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:49.846894 29453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:49.847009 29453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:49.857502 29453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:41137,127.6.215.253:33905,127.6.215.252:38627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab
--principal=kudu/127.6.215.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:33905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.215.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:49.858878 29453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:49.860220 29453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:49.872366 29458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:49.872336 29459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:49.872318 29461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:49.873214 29453 server_base.cc:1034] running on GCE node
I20260926 19:38:49.873996 29453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:49.875505 29453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:49.876749 29453 hybrid_clock.cc:648] HybridClock initialized: now 1790451529876672 us; error 59 us; skew 500 ppm
I20260926 19:38:49.877138 29453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:49 dist-test-slave-phbh krb5kdc[29366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:49.880236 29453 init.cc:377] Logged in from keytab as kudu/127.6.215.253@KRBTEST.COM (short username kudu)
I20260926 19:38:49.881764 29453 webserver.cc:460] Webserver started at http://127.6.215.253:41673/ using document root <none> and password file <none>
I20260926 19:38:49.882452 29453 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:49.882584 29453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:49.882908 29453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:49.885948 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data/instance:
uuid: "e227ee105b9e48f1b3c7c40645ee736d"
format_stamp: "Formatted at 2026-09-26 19:38:49 on dist-test-slave-phbh"
server_key: "2185b62b9eb463357217a44c670dccad"
server_key_iv: "be5664e3f907f51fb54a7e12a02165f5"
server_key_version: "encryptionkey@0"
I20260926 19:38:49.886730 29453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/wal/instance:
uuid: "e227ee105b9e48f1b3c7c40645ee736d"
format_stamp: "Formatted at 2026-09-26 19:38:49 on dist-test-slave-phbh"
server_key: "2185b62b9eb463357217a44c670dccad"
server_key_iv: "be5664e3f907f51fb54a7e12a02165f5"
server_key_version: "encryptionkey@0"
I20260926 19:38:49.892016 29453 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 19:38:49.895665 29468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:49.897125 29453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 19:38:49.897326 29453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/wal
uuid: "e227ee105b9e48f1b3c7c40645ee736d"
format_stamp: "Formatted at 2026-09-26 19:38:49 on dist-test-slave-phbh"
server_key: "2185b62b9eb463357217a44c670dccad"
server_key_iv: "be5664e3f907f51fb54a7e12a02165f5"
server_key_version: "encryptionkey@0"
I20260926 19:38:49.897557 29453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:49.911485 29453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:49.915591 29453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:49.915865 29453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:49.934808 29453 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.253:33905
I20260926 19:38:49.934849 29519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.253:33905 every 8 connection(s)
I20260926 19:38:49.936200 29453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
I20260926 19:38:49.940306 29520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:49.940855 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29453
I20260926 19:38:49.941026 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-1/wal/instance
I20260926 19:38:49.941504 7007 external_mini_cluster.cc:1428] Setting key 0baf9c01b49e491f583d8e664d27e687
I20260926 19:38:49.947578 29520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:49 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.252@KRBTEST.COM" created.
I20260926 19:38:49.969352 29448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.950548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.6.215.254:41137)
0926 19:38:49.950753 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:49.950758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:49.952255 (+ 1497us) server_negotiation.cc:407] Connection header received
0926 19:38:49.954684 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:49.954691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:49.954790 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:49.954909 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:49.956534 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.957290 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.958831 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.959245 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.959377 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:49.963284 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:49.963290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:49.963293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:49.963326 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:49.966153 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:49.966912 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:49.966919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:49.966923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:49.967097 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:49.967545 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:49.967553 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:49.967558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:49.967914 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:49.968071 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:49.968757 (+ 686us) server_negotiation.cc:299] Negotiation successful
0926 19:38:49.969000 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 19:38:49.970335 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.951199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41137 (local address 127.0.0.1:37028)
0926 19:38:49.952057 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.952096 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:49.954085 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:49.955173 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:49.955182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:49.955721 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:49.956352 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:49.956371 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.957453 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.957457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:49.958668 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:49.958681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.959513 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.959523 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:49.959745 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:49.960436 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:49.960459 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:49.962994 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:49.966321 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:49.966330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:49.966348 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:49.966793 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:49.967238 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:49.967246 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:49.967250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:49.967426 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:49.968045 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:49.968053 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:49.968397 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 19:38:49.968662 (+ 265us) client_negotiation.cc:240] Negotiation successful
0926 19:38:49.969228 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":395,"threads_started":1}
I20260926 19:38:49.972076 29520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:49 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:38:49.987897 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.973317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.6.215.253:33905)
0926 19:38:49.974049 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:49.974055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:49.974080 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:38:49.974121 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:49.974127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:49.974312 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:49.974473 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:49.975245 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.976352 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.977544 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.978122 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.978261 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:49.981872 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:49.981884 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:49.981898 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:49.981929 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:49.984847 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:49.985667 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:49.985673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:49.985676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:49.985789 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:49.986268 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:49.986273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:49.986275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:49.986658 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:49.986841 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:49.987322 (+ 481us) server_negotiation.cc:299] Negotiation successful
0926 19:38:49.987590 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":585,"thread_start_us":433,"threads_started":1}
I20260926 19:38:49.988008 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.972994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50610)
0926 19:38:49.973215 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.973234 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:49.973348 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:49.974455 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:49.974459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:49.974939 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:49.975070 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:49.975079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.976481 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.976489 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:49.977413 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:49.977424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:49.978270 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:49.978275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:49.978510 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:49.979084 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:49.979100 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:49.981619 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:49.984973 (+ 3354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:49.984981 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:49.984988 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:49.985518 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:49.985914 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:49.985922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:49.985929 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:49.986117 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:49.986799 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:49.986807 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:49.986979 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 19:38:49.987132 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 19:38:49.987475 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":37}
I20260926 19:38:49.991132 29520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:38:49.992431 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.991953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55790)
0926 19:38:49.992163 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.992276 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 19:38:49.992698 29471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
I20260926 19:38:49.994589 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:49.994114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55802)
0926 19:38:49.994345 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:49.994469 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260926 19:38:49.994801 29520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab.
Entry for principal kudu/127.6.215.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab.
I20260926 19:38:50.001554 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:38627
--webserver_interface=127.6.215.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab
--principal=kudu/127.6.215.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:34321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:41137,127.6.215.253:33905,127.6.215.252:38627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:38:50.044077 29520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 1
I20260926 19:38:50.045290 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.044838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55816)
0926 19:38:50.045025 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.045135 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 19:38:50.046324 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.045979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55818)
0926 19:38:50.046151 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.046223 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 19:38:50.046535 29520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
I20260926 19:38:50.114750 29520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 2
I20260926 19:38:50.115895 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.115426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55832)
0926 19:38:50.115625 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.115738 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 19:38:50.116930 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.116541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55846)
0926 19:38:50.116690 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.116758 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 19:38:50.117125 29520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
I20260926 19:38:50.138929 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } attempt: 5
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:38:50.156080 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.139670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50620)
0926 19:38:50.139895 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.139917 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:50.140057 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:50.140444 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:50.140452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:50.140914 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:50.141092 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.141101 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.142095 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.142104 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.142963 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.142974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.143702 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.143708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.143909 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.148326 (+ 4417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:50.148346 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:50.150491 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:50.153628 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.153634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.153638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.154016 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.154398 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.154402 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.154406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.154548 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.155096 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:50.155101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:50.155264 (+ 163us) client_negotiation.cc:769] Sending connection context
0926 19:38:50.155421 (+ 157us) client_negotiation.cc:240] Negotiation successful
0926 19:38:50.155734 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 19:38:50.156077 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.139770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.6.215.253:33905)
0926 19:38:50.139964 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.139971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.139993 (+ 22us) server_negotiation.cc:407] Connection header received
0926 19:38:50.140220 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.140226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.140294 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.140404 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.141266 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.141967 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.143119 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.143552 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.143679 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.150708 (+ 7029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.150720 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.150723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.150756 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.153472 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.154146 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.154160 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.154162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.154283 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.154661 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.154667 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.154671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.154973 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.155132 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.155526 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.155756 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 19:38:50.157452 29446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:38:50.158691 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.158319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55860)
0926 19:38:50.158484 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.158559 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 19:38:50.159606 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.159290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55862)
0926 19:38:50.159437 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.159511 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 19:38:50.159802 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
I20260926 19:38:50.208328 29520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 3
I20260926 19:38:50.209628 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.208991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55864)
0926 19:38:50.209245 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.209388 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 19:38:50.210968 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.210370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55866)
0926 19:38:50.210646 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.210769 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260926 19:38:50.211023 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 1
W20260926 19:38:50.211432 29520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
I20260926 19:38:50.212018 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.211599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55878)
0926 19:38:50.211766 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.211860 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260926 19:38:50.212944 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.212601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55882)
0926 19:38:50.212752 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.212849 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 19:38:50.213138 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
I20260926 19:38:50.281374 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 2
I20260926 19:38:50.282585 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.282034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55884)
0926 19:38:50.282334 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.282423 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 19:38:50.283589 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.283248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55898)
0926 19:38:50.283392 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.283478 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 19:38:50.283808 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
W20260926 19:38:50.332432 29527 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 19:38:50.332844 29527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:50.332973 29527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:50.333106 29527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:50.340536 29527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:50.340674 29527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:50.340816 29527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:50.340947 29527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:50.341058 29527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:50.351083 29527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:41137,127.6.215.253:33905,127.6.215.252:38627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab
--principal=kudu/127.6.215.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:38627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.215.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:50.352510 29527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:50.353951 29527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 19:38:50.354671 29520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 4
I20260926 19:38:50.355983 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.355376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55900)
0926 19:38:50.355656 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.355763 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 19:38:50.357179 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.356745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55914)
0926 19:38:50.356967 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.357056 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260926 19:38:50.357496 29520 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
W20260926 19:38:50.366317 29532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:50.367058 29533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:50.367410 29535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:50.368633 29527 server_base.cc:1034] running on GCE node
I20260926 19:38:50.369493 29527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:50.370919 29527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:50.372172 29527 hybrid_clock.cc:648] HybridClock initialized: now 1790451530372092 us; error 58 us; skew 500 ppm
I20260926 19:38:50.372506 29527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451530, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:50.375733 29527 init.cc:377] Logged in from keytab as kudu/127.6.215.252@KRBTEST.COM (short username kudu)
I20260926 19:38:50.377364 29527 webserver.cc:460] Webserver started at http://127.6.215.252:35225/ using document root <none> and password file <none>
I20260926 19:38:50.378079 29527 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:50.378211 29527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:50.378623 29527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:50.383031 29527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data/instance:
uuid: "fbcb49492e824cc7ab701685f7fe63dd"
format_stamp: "Formatted at 2026-09-26 19:38:50 on dist-test-slave-phbh"
server_key: "426ef52f9f3e7a9e0c958e0c95c7bd6a"
server_key_iv: "affd855c3596bce3e10e98f4b557a5d7"
server_key_version: "encryptionkey@0"
I20260926 19:38:50.384130 29527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/wal/instance:
uuid: "fbcb49492e824cc7ab701685f7fe63dd"
format_stamp: "Formatted at 2026-09-26 19:38:50 on dist-test-slave-phbh"
server_key: "426ef52f9f3e7a9e0c958e0c95c7bd6a"
server_key_iv: "affd855c3596bce3e10e98f4b557a5d7"
server_key_version: "encryptionkey@0"
I20260926 19:38:50.390002 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 3
I20260926 19:38:50.391534 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.390924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55924)
0926 19:38:50.391183 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.391316 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 19:38:50.392091 29527 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260926 19:38:50.393373 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.392498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55940)
0926 19:38:50.392846 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.392954 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
W20260926 19:38:50.393699 29446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:38627: Network error: Client connection negotiation failed: client connection to 127.6.215.252:38627: connect: Connection refused (error 111)
I20260926 19:38:50.397351 29542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:50.399216 29527 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 19:38:50.399408 29527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/wal
uuid: "fbcb49492e824cc7ab701685f7fe63dd"
format_stamp: "Formatted at 2026-09-26 19:38:50 on dist-test-slave-phbh"
server_key: "426ef52f9f3e7a9e0c958e0c95c7bd6a"
server_key_iv: "affd855c3596bce3e10e98f4b557a5d7"
server_key_version: "encryptionkey@0"
I20260926 19:38:50.399693 29527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:50.432217 29527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:50.437036 29527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:50.437327 29527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:50.455719 29527 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.252:38627
I20260926 19:38:50.455742 29593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.252:38627 every 8 connection(s)
I20260926 19:38:50.457235 29527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
I20260926 19:38:50.461179 29594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:50.466569 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29527
I20260926 19:38:50.466771 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790451176367082-7007-0/minicluster-data/master-2/wal/instance
I20260926 19:38:50.467222 7007 external_mini_cluster.cc:1428] Setting key 6844df05b51450b426bfa426bfed9740
I20260926 19:38:50.468139 29594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451530, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
I20260926 19:38:50.488713 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.471376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.6.215.254:41137)
0926 19:38:50.472893 (+ 1517us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.472897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.472915 (+ 18us) server_negotiation.cc:407] Connection header received
0926 19:38:50.473256 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.473261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.473355 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.473486 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.474878 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.475711 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.477467 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.477955 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.478051 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.482058 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.482067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.482073 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.482115 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.485022 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.485928 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.485932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.485934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.486056 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.486543 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.486548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.486551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.486927 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.487064 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.487953 (+ 889us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.488277 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1360,"thread_start_us":1229,"threads_started":1}
I20260926 19:38:50.490057 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.471380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:41137 (local address 127.0.0.1:37042)
0926 19:38:50.472200 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.472242 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:50.473004 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:50.473593 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:50.473601 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:50.474322 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:50.474684 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.474713 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.475872 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.475878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.477329 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.477340 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.478192 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.478202 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.478532 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.479387 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:50.479416 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:50.481836 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:50.485155 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.485165 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.485178 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.485795 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.486165 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.486171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.486174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.486419 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.487043 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:50.487051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:50.487548 (+ 497us) client_negotiation.cc:769] Sending connection context
0926 19:38:50.487856 (+ 308us) client_negotiation.cc:240] Negotiation successful
0926 19:38:50.488387 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":42,"thread_start_us":382,"threads_started":1}
I20260926 19:38:50.490077 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.469320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.6.215.252:38627)
0926 19:38:50.470086 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.470101 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.470147 (+ 46us) server_negotiation.cc:407] Connection header received
0926 19:38:50.472991 (+ 2844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.473013 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.473263 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.473425 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.474963 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.476331 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.477647 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.478284 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.478422 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.482627 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.482635 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.482648 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.482674 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.486194 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.487029 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.487037 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.487040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.487189 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.487557 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.487563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.487569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.487991 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.488284 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.488677 (+ 393us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.488958 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":543,"thread_start_us":328,"threads_started":1}
I20260926 19:38:50.491925 29594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451530, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:50.507844 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.492785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:33905 (local address 127.0.0.1:50636)
0926 19:38:50.493127 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.493146 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:50.493318 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:50.493773 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:50.493776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:50.494120 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:50.494359 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.494374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.495791 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.495796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.497165 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.497176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.498298 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.498303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.498469 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.499034 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:50.499059 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:50.501885 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:50.505236 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.505244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.505251 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.505839 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.506326 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.506329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.506332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.506478 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.507099 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:50.507103 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:50.507231 (+ 128us) client_negotiation.cc:769] Sending connection context
0926 19:38:50.507379 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 19:38:50.507589 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260926 19:38:50.508149 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.492758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.6.215.253:33905)
0926 19:38:50.492993 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.492997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.493245 (+ 248us) server_negotiation.cc:407] Connection header received
0926 19:38:50.493546 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.493552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.493639 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.493751 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.494534 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.495659 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.497435 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.498087 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.498233 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.502117 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.502123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.502126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.502161 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.505077 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.506018 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.506022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.506024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.506200 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.506604 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.506609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.506612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.506981 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.507129 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.507524 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.507748 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 19:38:50.509172 29594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:38:50.511329 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.493425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.6.215.254:41137)
0926 19:38:50.494018 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.494024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.494039 (+ 15us) server_negotiation.cc:407] Connection header received
0926 19:38:50.494087 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.494090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.494153 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.494257 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.495779 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.496904 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.498684 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.499309 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.499448 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.504042 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.504056 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.504060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.504093 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.507861 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.508799 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.508806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.508810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.508940 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.509323 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.509328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.509332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.509617 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.509904 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.510874 (+ 970us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.511048 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457}
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451530, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:38:50.523746 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.510053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55960 (local address 127.6.215.252:38627)
0926 19:38:50.510211 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.510215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.510336 (+ 121us) server_negotiation.cc:407] Connection header received
0926 19:38:50.510487 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.510490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.510541 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.510613 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.511544 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.512445 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.513732 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.514133 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.514297 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.517609 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.517614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.517617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.517645 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.520962 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.521643 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.521647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.521649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.521778 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.522212 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.522217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.522220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.522532 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.522683 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.523058 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.523454 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":53}
I20260926 19:38:50.523753 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.510107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55960)
0926 19:38:50.510273 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.510292 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:50.510393 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:50.510671 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:50.510677 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:50.511138 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:50.511323 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.511332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.512564 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.512572 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.513602 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.513613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.514310 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.514316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.514543 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.515107 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:50.515132 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:50.517437 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:50.521086 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.521090 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.521092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.521525 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.521890 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.521895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.521899 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.522068 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.522657 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:50.522662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:50.522830 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 19:38:50.522973 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 19:38:50.523325 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":49}
I20260926 19:38:50.528632 29594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: Bootstrap starting.
I20260926 19:38:50.528975 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.513182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.6.215.253:33905)
0926 19:38:50.513388 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.513393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.513410 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:38:50.513490 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.513496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.513564 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.513662 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.514860 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.515997 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.517337 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.517854 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.517947 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.523107 (+ 5160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.523112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.523116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.523147 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.526302 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.526979 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.526984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.526988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.527124 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.527500 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.527506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.527509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.527764 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.528037 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.528347 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.528574 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 19:38:50.532558 29594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:50.533946 29594 log.cc:824] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:50.534406 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:50.537272 29594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: No bootstrap required, opened a new log
I20260926 19:38:50.537755 29421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37052
I20260926 19:38:50.544516 29594 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.544965 29594 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbcb49492e824cc7ab701685f7fe63dd, State: Initialized, Role: FOLLOWER
I20260926 19:38:50.545425 29594 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.546311 29601 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.546623 29601 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:50.547631 29594 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [sys.catalog]: configured and running, proceeding with master startup.
W20260926 19:38:50.557253 29612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:38:50.557394 29612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:38:50.554728 29527 master_runner.cc:186] Error getting master registration for 127.6.215.254:41137: OK, instance_id { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" instance_seqno: 1790451529429107 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:38:50.563511 29527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:38:50.570948 29446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 4
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
I20260926 19:38:50.585042 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.571783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.6.215.252:38627)
0926 19:38:50.571986 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.571992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.572013 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:38:50.572190 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.572196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.572282 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.572378 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.573235 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.574006 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.575303 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.575781 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.575922 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.579294 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.579299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.579302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.579346 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.582196 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.582889 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.582897 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.582905 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.583028 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.583473 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.583479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.583483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.583842 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.584022 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.584402 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.584616 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 19:38:50.585037 29449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.571679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55962)
0926 19:38:50.571869 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.571893 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:50.572022 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:50.572426 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:50.572432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:50.572916 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:50.573083 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.573092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.574171 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.574179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.575143 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.575156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.575938 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.575944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.576167 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.576860 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:50.576878 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:50.579094 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:50.582367 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.582373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.582377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.582756 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.583146 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.583151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.583154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.583356 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.583977 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:50.583982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:50.584156 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 19:38:50.584314 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 19:38:50.584616 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 19:38:50.591320 29446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7: Bootstrap starting.
I20260926 19:38:50.595850 29446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:50.597332 29446 log.cc:824] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:50.601123 29446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7: No bootstrap required, opened a new log
I20260926 19:38:50.609539 29446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.610030 29446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ef5f8d6b2a4b8094b9b033618c04b7, State: Initialized, Role: FOLLOWER
I20260926 19:38:50.610633 29446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.611729 29615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.612133 29615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:50.613139 29446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:50.613211 29421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 19:38:50.613452 29421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:50.616298 29421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.617666 29421 leader_election.cc:288] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [CANDIDATE]: Term 1 election: Requested vote from peers e227ee105b9e48f1b3c7c40645ee736d (127.6.215.253:33905), fbcb49492e824cc7ab701685f7fe63dd (127.6.215.252:38627)
I20260926 19:38:50.619778 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fbcb49492e824cc7ab701685f7fe63dd"
I20260926 19:38:50.620345 29569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:50.625479 29569 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef5f8d6b2a4b8094b9b033618c04b7 in term 1.
I20260926 19:38:50.626691 29397 leader_election.cc:302] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83ef5f8d6b2a4b8094b9b033618c04b7, fbcb49492e824cc7ab701685f7fe63dd; no voters:
I20260926 19:38:50.627365 29615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:50.639731 29615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [term 1 LEADER]: Becoming Leader. State: Replica: 83ef5f8d6b2a4b8094b9b033618c04b7, State: Running, Role: LEADER
I20260926 19:38:50.641362 29615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.656730 29520 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } attempt: 5
I20260926 19:38:50.657788 29624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ef5f8d6b2a4b8094b9b033618c04b7. Latest consensus state: current_term: 1 leader_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.658260 29624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: This master's current role is: LEADER
W20260926 19:38:50.668349 29400 tablet.cc:2369] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 19:38:50 dist-test-slave-phbh krb5kdc[29366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451529, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
I20260926 19:38:50.674508 29634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 19:38:50.681739 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.658319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55978 (local address 127.6.215.252:38627)
0926 19:38:50.658554 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.658562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.658583 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:38:50.658659 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.658667 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.658775 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.658892 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.660286 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.661690 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.663962 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.664903 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.665050 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.671635 (+ 6585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.671645 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.671653 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.671703 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.675738 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.676645 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.676651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.676654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.676968 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.677706 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.677711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.677715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.678392 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.679097 (+ 705us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.679627 (+ 530us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.680412 (+ 785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:38:50.681907 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.657423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:38627 (local address 127.0.0.1:55978)
0926 19:38:50.657862 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:50.657887 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:50.658045 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:50.659121 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:50.659130 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:50.659767 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:50.660034 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.660051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.661902 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.661912 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.663749 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:50.663768 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.665297 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.665305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:50.665620 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.666376 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:50.666407 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:50.671391 (+ 4984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:50.675937 (+ 4546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.675942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.675946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.676491 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.677217 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:50.677221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:50.677225 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:50.677527 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:50.678601 (+ 1074us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:50.678606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:50.678925 (+ 319us) client_negotiation.cc:769] Sending connection context
0926 19:38:50.679580 (+ 655us) client_negotiation.cc:240] Negotiation successful
0926 19:38:50.680609 (+ 1029us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260926 19:38:50.688058 29520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d: Bootstrap starting.
W20260926 19:38:50.689551 29548 tablet.cc:2369] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:50.691862 29634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 19:38:50.692119 29520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:50.693851 29520 log.cc:824] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:50.697278 29520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d: No bootstrap required, opened a new log
I20260926 19:38:50.708485 29520 raft_consensus.cc:348] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.708915 29520 raft_consensus.cc:729] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e227ee105b9e48f1b3c7c40645ee736d, State: Initialized, Role: FOLLOWER
I20260926 19:38:50.709450 29520 consensus_queue.cc:260] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } }
I20260926 19:38:50.679150 29379 master_runner.cc:186] Error getting master registration for 127.6.215.253:33905: OK, instance_id { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" instance_seqno: 1790451529929445 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:38:50.718909 29379 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:38:50.720892 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.662718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.6.215.252:38627)
0926 19:38:50.668314 (+ 5596us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.668537 (+ 223us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.674016 (+ 5479us) server_negotiation.cc:407] Connection header received
0926 19:38:50.674098 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.674106 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.674201 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.674339 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.693260 (+ 18921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.696897 (+ 3637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.698668 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.699975 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.700961 (+ 986us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.708672 (+ 7711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.708679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.708684 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.708733 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.714396 (+ 5663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.715902 (+ 1506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.715912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.715915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.716246 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.717127 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.717161 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.717167 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.717829 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.718233 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.719707 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.720102 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4890,"thread_start_us":3651,"threads_started":1}
I20260926 19:38:50.723111 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.656402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.6.215.254:41137)
0926 19:38:50.656646 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.656652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.668459 (+ 11807us) server_negotiation.cc:407] Connection header received
0926 19:38:50.669439 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.669447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.669543 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.669629 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.696817 (+ 27188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.697950 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.708638 (+ 10688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.709367 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.709494 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.712713 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.712721 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.712726 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.712789 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.716435 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.719101 (+ 2666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.719108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.719111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.719318 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.719780 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.719785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.719788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.720259 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.720525 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.722458 (+ 1933us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.722670 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 19:38:50.724627 29569 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [term 1 FOLLOWER]: Refusing update from remote peer 83ef5f8d6b2a4b8094b9b033618c04b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:50.726102 29624 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:50.733775 29601 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ef5f8d6b2a4b8094b9b033618c04b7. Latest consensus state: current_term: 1 leader_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.734107 29601 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:50.735833 29520 sys_catalog.cc:564] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:50.736986 29495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e227ee105b9e48f1b3c7c40645ee736d"
I20260926 19:38:50.737526 29495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:50.741197 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.656442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.6.215.253:33905)
0926 19:38:50.656831 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.656838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.670860 (+ 14022us) server_negotiation.cc:407] Connection header received
0926 19:38:50.671364 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.671373 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.671469 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.671547 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.690921 (+ 19374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.692069 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.695606 (+ 3537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.696313 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.696451 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:50.721104 (+ 24653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:50.721112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:50.721117 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:50.721188 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:50.736552 (+ 15364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.737558 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.737564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.737567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.737755 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:50.738216 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:50.738221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:50.738225 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:50.738604 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:50.740280 (+ 1676us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:50.740421 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 19:38:50.740702 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260926 19:38:50.743634 29495 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ef5f8d6b2a4b8094b9b033618c04b7 in term 1.
I20260926 19:38:50.744367 29494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [term 1 FOLLOWER]: Refusing update from remote peer 83ef5f8d6b2a4b8094b9b033618c04b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:50.746039 29624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.746587 29615 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:50.747051 29624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:50.748080 29640 mvcc.cc:205] Tried to move back new op lower bound from 7333689469781516288 to 7333689469524807680. Current Snapshot: MvccSnapshot[applied={T|T < 7333689469781516288}]
I20260926 19:38:50.721947 29641 sys_catalog.cc:455] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.748490 29641 sys_catalog.cc:458] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:50.755149 29634 catalog_manager.cc:1347] Generated new cluster ID: c30e833ecb484d1ba8d30403b0cc08a1
I20260926 19:38:50.755260 29634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:50.770857 29601 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.771124 29601 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:50.781193 29634 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:50.783231 29634 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:50.806114 29651 sys_catalog.cc:455] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.806753 29651 sys_catalog.cc:458] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:50.807112 29641 sys_catalog.cc:455] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ef5f8d6b2a4b8094b9b033618c04b7. Latest consensus state: current_term: 1 leader_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.807344 29641 sys_catalog.cc:458] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:50.808607 29615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ef5f8d6b2a4b8094b9b033618c04b7" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 41137 } } peers { permanent_uuid: "e227ee105b9e48f1b3c7c40645ee736d" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 33905 } } peers { permanent_uuid: "fbcb49492e824cc7ab701685f7fe63dd" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 38627 } } }
I20260926 19:38:50.808971 29615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7 [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:50.818514 29634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83ef5f8d6b2a4b8094b9b033618c04b7: Generated new TSK 0
I20260926 19:38:50.819901 29634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 19:38:50.908856 29662 catalog_manager.cc:1259] Loaded cluster ID: c30e833ecb484d1ba8d30403b0cc08a1
I20260926 19:38:50.908969 29662 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d: loading cluster ID for follower catalog manager: success
I20260926 19:38:50.918958 29662 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d: acquiring CA information for follower catalog manager: success
I20260926 19:38:50.923568 29662 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e227ee105b9e48f1b3c7c40645ee736d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 19:38:50.955646 29599 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:50.941239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37070 (local address 127.6.215.254:41137)
0926 19:38:50.941489 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.941498 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.948221 (+ 6723us) server_negotiation.cc:407] Connection header received
0926 19:38:50.949183 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.949198 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.949343 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.949438 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.955317 (+ 5879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37070: BlockingRecv error: recv got EOF from 127.0.0.1:37070 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 19:38:50.958413 29524 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:50.942544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.6.215.253:33905)
0926 19:38:50.942894 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.942900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.956387 (+ 13487us) server_negotiation.cc:407] Connection header received
0926 19:38:50.956688 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.956697 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.956857 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.956946 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.958136 (+ 1190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50674: BlockingRecv error: recv got EOF from 127.0.0.1:50674 (error 108)
Metrics: {"server-negotiator.queue_time_us":217}
W20260926 19:38:50.961263 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:50.942704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.6.215.252:38627)
0926 19:38:50.943050 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.943056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.959071 (+ 16015us) server_negotiation.cc:407] Connection header received
0926 19:38:50.959349 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.959357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.959463 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.959550 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:50.960888 (+ 1338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55996: BlockingRecv error: recv got EOF from 127.0.0.1:55996 (error 108)
Metrics: {"server-negotiator.queue_time_us":216}
I20260926 19:38:51.002208 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:50.982892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.6.215.253:33905)
0926 19:38:50.983077 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:50.983106 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:50.993915 (+ 10809us) server_negotiation.cc:407] Connection header received
0926 19:38:50.994538 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:50.994545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:50.994774 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:50.994908 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:50.995638 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.996963 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.999025 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:50.999702 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:50.999809 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.000540 (+ 731us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.001317 (+ 777us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.001434 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.001755 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.001859 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 19:38:51.022222 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.010028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.6.215.252:38627)
0926 19:38:51.010211 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.010217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.019913 (+ 9696us) server_negotiation.cc:407] Connection header received
0926 19:38:51.020257 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.020263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.020416 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.020501 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.021685 (+ 1184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56006: BlockingRecv error: recv got EOF from 127.0.0.1:56006 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 19:38:51.081907 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.063705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.6.215.253:33905)
0926 19:38:51.063899 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.063905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.074536 (+ 10631us) server_negotiation.cc:407] Connection header received
0926 19:38:51.075120 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.075127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.075222 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.075290 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.076013 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.077244 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.079412 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.080174 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.080274 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.080938 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.081101 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.081186 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.081427 (+ 241us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.081530 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 19:38:51.122834 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.112207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.6.215.252:38627)
0926 19:38:51.112419 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.112426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.120816 (+ 8390us) server_negotiation.cc:407] Connection header received
0926 19:38:51.121222 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.121229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.121355 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.121437 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.122553 (+ 1116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56016: BlockingRecv error: recv got EOF from 127.0.0.1:56016 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 19:38:51.180225 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.161946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50702 (local address 127.6.215.253:33905)
0926 19:38:51.162139 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.162147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.173138 (+ 10991us) server_negotiation.cc:407] Connection header received
0926 19:38:51.173672 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.173679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.173781 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.173847 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.174549 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.175760 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.177907 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.178598 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.178694 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.179305 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.179474 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.179543 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.179783 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.179872 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260926 19:38:51.220943 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.211934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56022 (local address 127.6.215.252:38627)
0926 19:38:51.212153 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.212159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.218884 (+ 6725us) server_negotiation.cc:407] Connection header received
0926 19:38:51.219310 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.219316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.219455 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.219544 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.220642 (+ 1098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56022: BlockingRecv error: recv got EOF from 127.0.0.1:56022 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:38:51.256620 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.249295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.6.215.253:33905)
0926 19:38:51.249483 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.249490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.250052 (+ 562us) server_negotiation.cc:407] Connection header received
0926 19:38:51.250272 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.250281 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.250399 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.250547 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.251106 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.252343 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.254397 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.255189 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.255392 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.255800 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.255960 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.256071 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.256236 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.256323 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 19:38:51.266189 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.263871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.6.215.252:38627)
0926 19:38:51.264098 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.264104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.264598 (+ 494us) server_negotiation.cc:407] Connection header received
0926 19:38:51.264760 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.264784 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.264892 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.265005 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.265944 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56034: BlockingRecv error: recv got EOF from 127.0.0.1:56034 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260926 19:38:51.290464 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.283213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.6.215.253:33905)
0926 19:38:51.283417 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.283423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.284045 (+ 622us) server_negotiation.cc:407] Connection header received
0926 19:38:51.284209 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.284217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.284321 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.284435 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.284974 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.286188 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.288229 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.288998 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.289127 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.289612 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.289792 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.289904 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.290059 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.290150 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260926 19:38:51.299073 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.297152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.6.215.252:38627)
0926 19:38:51.297337 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.297345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.297491 (+ 146us) server_negotiation.cc:407] Connection header received
0926 19:38:51.297768 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.297773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.297862 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.297967 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.298833 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56036: BlockingRecv error: recv got EOF from 127.0.0.1:56036 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 19:38:51.324982 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.318936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50732 (local address 127.6.215.253:33905)
0926 19:38:51.319144 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.319150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.319543 (+ 393us) server_negotiation.cc:407] Connection header received
0926 19:38:51.319741 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.319745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.319830 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.319917 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.320375 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.321252 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.323189 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.323607 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.323722 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.324212 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.324323 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.324427 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.324577 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.324651 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 19:38:51.333489 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.331623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.6.215.252:38627)
0926 19:38:51.331835 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.331841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.332105 (+ 264us) server_negotiation.cc:407] Connection header received
0926 19:38:51.332255 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.332263 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.332363 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.332480 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.333259 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56042: BlockingRecv error: recv got EOF from 127.0.0.1:56042 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 19:38:51.374957 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.368954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50742 (local address 127.6.215.253:33905)
0926 19:38:51.369192 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.369200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.369537 (+ 337us) server_negotiation.cc:407] Connection header received
0926 19:38:51.369705 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.369715 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.369844 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.369958 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.370325 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.371503 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.372939 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.373654 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.373812 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.374180 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.374316 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.374422 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.374522 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.374599 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260926 19:38:51.383064 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.381094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.6.215.252:38627)
0926 19:38:51.381316 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.381322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.381643 (+ 321us) server_negotiation.cc:407] Connection header received
0926 19:38:51.381842 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.381848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.381956 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.382071 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.382814 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56056: BlockingRecv error: recv got EOF from 127.0.0.1:56056 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 19:38:51.456720 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.450760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50754 (local address 127.6.215.253:33905)
0926 19:38:51.451003 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.451012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.451524 (+ 512us) server_negotiation.cc:407] Connection header received
0926 19:38:51.451767 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.451772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.451849 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.451958 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.452376 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.453214 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.454540 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.455182 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.455326 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.455905 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.456056 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.456123 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.456349 (+ 226us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.456428 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260926 19:38:51.469120 29636 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:51.466833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.6.215.252:38627)
0926 19:38:51.467073 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.467083 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.467629 (+ 546us) server_negotiation.cc:407] Connection header received
0926 19:38:51.467840 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.467846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.467926 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.468032 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:51.468903 (+ 871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56058: BlockingRecv error: recv got EOF from 127.0.0.1:56058 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 19:38:51.559392 29612 catalog_manager.cc:1259] Loaded cluster ID: c30e833ecb484d1ba8d30403b0cc08a1
I20260926 19:38:51.559492 29612 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: loading cluster ID for follower catalog manager: success
I20260926 19:38:51.561975 29612 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: acquiring CA information for follower catalog manager: success
I20260926 19:38:51.563711 29612 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fbcb49492e824cc7ab701685f7fe63dd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:38:51.606365 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.600473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.6.215.253:33905)
0926 19:38:51.600752 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.600759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.601067 (+ 308us) server_negotiation.cc:407] Connection header received
0926 19:38:51.601304 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.601310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.601383 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.601470 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.601898 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.603094 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.604463 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.605011 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.605142 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.605571 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.605739 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.605835 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.605981 (+ 146us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.606060 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260926 19:38:51.617934 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.612126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.6.215.252:38627)
0926 19:38:51.612382 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.612389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.612650 (+ 261us) server_negotiation.cc:407] Connection header received
0926 19:38:51.612946 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.612952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.613113 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.613244 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.613668 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.614480 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.615764 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.616150 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.616291 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.616816 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.617355 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.617514 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.617570 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.617658 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 19:38:51.634931 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.624330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.6.215.254:41137)
0926 19:38:51.627870 (+ 3540us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.627879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.628384 (+ 505us) server_negotiation.cc:407] Connection header received
0926 19:38:51.628687 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.628695 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.628959 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.629152 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.629783 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.630730 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.632327 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.632723 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.632892 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.633368 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.633693 (+ 325us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.633815 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.634642 (+ 827us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.634708 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3378,"thread_start_us":3261,"threads_started":1}
I20260926 19:38:51.635898 29524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.624868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.6.215.253:33905)
0926 19:38:51.625112 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.625121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.628806 (+ 3685us) server_negotiation.cc:407] Connection header received
0926 19:38:51.629070 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.629074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.629179 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.629270 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.630179 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.631322 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.633940 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.634469 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.634564 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.635112 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.635237 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.635299 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.635524 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.635617 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 19:38:51.635941 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:51.624645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.6.215.252:38627)
0926 19:38:51.625116 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:51.625125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:51.629157 (+ 4032us) server_negotiation.cc:407] Connection header received
0926 19:38:51.629417 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:51.629422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:51.629492 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:51.629590 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:51.630331 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.631364 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.633961 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:51.634513 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:51.634639 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:51.635201 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:51.635349 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:51.635408 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:51.635608 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 19:38:51.635681 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260926 19:38:51.640592 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29379
I20260926 19:38:51.641721 29441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:51.751988 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29379
I20260926 19:38:51.773268 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29453
I20260926 19:38:51.774335 29515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:51.882865 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29453
I20260926 19:38:51.903424 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29527
I20260926 19:38:51.904554 29589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:52.011475 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29527
2026-09-26T19:38:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4399 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 19:38:52 dist-test-slave-phbh krb5kdc[29776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 19:38:52 dist-test-slave-phbh krb5kdc[29776](info): set up 2 sockets
Sep 26 19:38:52 dist-test-slave-phbh krb5kdc[29776](info): commencing operation
krb5kdc: starting...
W20260926 19:38:53.129581 7007 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/test-user.keytab.
Sep 26 19:38:53 dist-test-slave-phbh krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T19:38:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T19:38:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.215.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
Entry for principal kudu/127.6.215.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab.
I20260926 19:38:53.399935 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40949
--webserver_interface=127.6.215.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:34949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:40949,127.6.215.253:46865,127.6.215.252:40901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 19:38:53.738567 29789 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 19:38:53.738922 29789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:53.739064 29789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:53.739181 29789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:53.746539 29789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:53.746631 29789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:53.746724 29789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:53.746794 29789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:53.746891 29789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:53.756933 29789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:40949,127.6.215.253:46865,127.6.215.252:40901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.254.keytab
--principal=kudu/127.6.215.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.254:40949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.215.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:53.758358 29789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:53.759732 29789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:53.771701 29797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:53.771751 29795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:53.772243 29794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:53.772538 29789 server_base.cc:1034] running on GCE node
I20260926 19:38:53.773420 29789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:53.774849 29789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:53.776131 29789 hybrid_clock.cc:648] HybridClock initialized: now 1790451533776063 us; error 37 us; skew 500 ppm
I20260926 19:38:53.776551 29789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:53 dist-test-slave-phbh krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:53.783512 29789 init.cc:377] Logged in from keytab as kudu/127.6.215.254@KRBTEST.COM (short username kudu)
I20260926 19:38:53.785125 29789 webserver.cc:460] Webserver started at http://127.6.215.254:34167/ using document root <none> and password file <none>
I20260926 19:38:53.785892 29789 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:53.786024 29789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:53.786367 29789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:53.789994 29789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data/instance:
uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b"
format_stamp: "Formatted at 2026-09-26 19:38:53 on dist-test-slave-phbh"
server_key: "2a22381dffa649017b104a73eabcbba9"
server_key_iv: "cc42aa13e39d5b93b6a08c364ee3cad7"
server_key_version: "encryptionkey@0"
I20260926 19:38:53.790796 29789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/wal/instance:
uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b"
format_stamp: "Formatted at 2026-09-26 19:38:53 on dist-test-slave-phbh"
server_key: "2a22381dffa649017b104a73eabcbba9"
server_key_iv: "cc42aa13e39d5b93b6a08c364ee3cad7"
server_key_version: "encryptionkey@0"
I20260926 19:38:53.795935 29789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 19:38:53.799499 29804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:53.801048 29789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 19:38:53.801255 29789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/wal
uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b"
format_stamp: "Formatted at 2026-09-26 19:38:53 on dist-test-slave-phbh"
server_key: "2a22381dffa649017b104a73eabcbba9"
server_key_iv: "cc42aa13e39d5b93b6a08c364ee3cad7"
server_key_version: "encryptionkey@0"
I20260926 19:38:53.801502 29789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:53.835008 29789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:53.839516 29789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:53.839809 29789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:53.859424 29789 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.254:40949
I20260926 19:38:53.859383 29855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.254:40949 every 8 connection(s)
I20260926 19:38:53.860919 29789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/data/info.pb
I20260926 19:38:53.864564 29856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:53.869786 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29789
I20260926 19:38:53.870007 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-0/wal/instance
I20260926 19:38:53.870440 7007 external_mini_cluster.cc:1428] Setting key 00081237d58c632b513a6059c0969183
I20260926 19:38:53.871127 29856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:53 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:53.893172 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:53.874303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40949 (local address 127.0.0.1:38650)
0926 19:38:53.875149 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:53.875200 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:53.877258 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:53.878066 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:53.878073 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:53.878751 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:53.879080 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:53.879097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:53.880635 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:53.880642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:53.881786 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:53.881798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:53.882715 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:53.882721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:53.882915 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:53.883595 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:53.883617 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:53.885788 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:53.888954 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:53.888967 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:53.888979 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:53.889361 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:53.889751 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:53.889757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:53.889764 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:53.889925 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:53.890662 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:53.890670 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:53.890919 (+ 249us) client_negotiation.cc:769] Sending connection context
0926 19:38:53.891154 (+ 235us) client_negotiation.cc:240] Negotiation successful
0926 19:38:53.891767 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":154,"thread_start_us":354,"threads_started":1}
I20260926 19:38:53.893208 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:53.874320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.6.215.254:40949)
0926 19:38:53.875923 (+ 1603us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:53.875936 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:53.875987 (+ 51us) server_negotiation.cc:407] Connection header received
0926 19:38:53.877617 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:53.877622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:53.877851 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:53.878045 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:53.879288 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:53.880482 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:53.881934 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:53.882480 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:53.882694 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:53.886022 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:53.886031 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:53.886042 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:53.886070 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:53.888837 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:53.889491 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:53.889505 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:53.889507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:53.889642 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:53.890036 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:53.890048 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:53.890051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:53.890532 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:53.890748 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:53.891284 (+ 536us) server_negotiation.cc:299] Negotiation successful
0926 19:38:53.891623 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1411,"thread_start_us":1179,"threads_started":1}
WARNING: no policy specified for kudu/127.6.215.253@KRBTEST.COM; defaulting to no policy
I20260926 19:38:53.896179 29856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.253@KRBTEST.COM" created.
I20260926 19:38:53.897276 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:53.896911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56360)
0926 19:38:53.897069 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:53.897168 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 19:38:53.897516 29806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
I20260926 19:38:53.899058 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:53.898688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56368)
0926 19:38:53.898878 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:53.898940 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 19:38:53.899242 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:46865: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab.
Entry for principal kudu/127.6.215.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab.
I20260926 19:38:53.928087 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:46865
--webserver_interface=127.6.215.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab
--principal=kudu/127.6.215.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:34949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:40949,127.6.215.253:46865,127.6.215.252:40901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:38:53.948562 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } attempt: 1
I20260926 19:38:53.949723 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:53.949268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56370)
0926 19:38:53.949473 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:53.949588 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 19:38:53.950846 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:53.950519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56386)
0926 19:38:53.950655 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:53.950722 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 19:38:53.951110 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:46865: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
I20260926 19:38:54.019361 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } attempt: 2
I20260926 19:38:54.020483 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.020001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56396)
0926 19:38:54.020254 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.020348 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 19:38:54.021632 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.021304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56398)
0926 19:38:54.021433 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.021510 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 19:38:54.021903 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:46865: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
I20260926 19:38:54.113171 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } attempt: 3
I20260926 19:38:54.114585 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.114060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56404)
0926 19:38:54.114355 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.114448 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260926 19:38:54.115763 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.115318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56406)
0926 19:38:54.115470 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.115549 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 19:38:54.115967 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:46865: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
W20260926 19:38:54.255666 29863 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 19:38:54.256026 29863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:54.256165 29863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:54.256284 29863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 19:38:54.259240 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } attempt: 4
I20260926 19:38:54.260592 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.259946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56422)
0926 19:38:54.260353 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.260448 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260926 19:38:54.261759 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.261386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56432)
0926 19:38:54.261540 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.261635 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 19:38:54.261969 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.253:46865: Network error: Client connection negotiation failed: client connection to 127.6.215.253:46865: connect: Connection refused (error 111)
W20260926 19:38:54.263916 29863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:54.263993 29863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:54.264087 29863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:54.264168 29863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:54.264259 29863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:54.274510 29863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:40949,127.6.215.253:46865,127.6.215.252:40901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.253.keytab
--principal=kudu/127.6.215.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.253:46865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.215.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:54.275851 29863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:54.277207 29863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:54.289389 29868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:54.289475 29869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:54.289651 29871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:54.291154 29863 server_base.cc:1034] running on GCE node
I20260926 19:38:54.291950 29863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:54.293520 29863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:54.294745 29863 hybrid_clock.cc:648] HybridClock initialized: now 1790451534294655 us; error 56 us; skew 500 ppm
I20260926 19:38:54.295073 29863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:54.298242 29863 init.cc:377] Logged in from keytab as kudu/127.6.215.253@KRBTEST.COM (short username kudu)
I20260926 19:38:54.299762 29863 webserver.cc:460] Webserver started at http://127.6.215.253:40547/ using document root <none> and password file <none>
I20260926 19:38:54.300418 29863 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:54.300503 29863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:54.300761 29863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:54.303719 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data/instance:
uuid: "ba5bae15240349a89603a589e0105a06"
format_stamp: "Formatted at 2026-09-26 19:38:54 on dist-test-slave-phbh"
server_key: "fce65ebd3da1bb03c923a2c426c5095c"
server_key_iv: "34ad310c86bece00ed0417004e05d3d5"
server_key_version: "encryptionkey@0"
I20260926 19:38:54.304498 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/wal/instance:
uuid: "ba5bae15240349a89603a589e0105a06"
format_stamp: "Formatted at 2026-09-26 19:38:54 on dist-test-slave-phbh"
server_key: "fce65ebd3da1bb03c923a2c426c5095c"
server_key_iv: "34ad310c86bece00ed0417004e05d3d5"
server_key_version: "encryptionkey@0"
I20260926 19:38:54.309755 29863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260926 19:38:54.313267 29878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:54.314653 29863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 19:38:54.314849 29863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/wal
uuid: "ba5bae15240349a89603a589e0105a06"
format_stamp: "Formatted at 2026-09-26 19:38:54 on dist-test-slave-phbh"
server_key: "fce65ebd3da1bb03c923a2c426c5095c"
server_key_iv: "34ad310c86bece00ed0417004e05d3d5"
server_key_version: "encryptionkey@0"
I20260926 19:38:54.315078 29863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:54.326571 29863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:54.330582 29863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:54.330853 29863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:54.349831 29863 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.253:46865
I20260926 19:38:54.349850 29929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.253:46865 every 8 connection(s)
I20260926 19:38:54.351236 29863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/data/info.pb
I20260926 19:38:54.353313 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29863
I20260926 19:38:54.353502 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-1/wal/instance
I20260926 19:38:54.354130 7007 external_mini_cluster.cc:1428] Setting key d6cc7497178b9129e30988ee0cef2376
I20260926 19:38:54.355962 29930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:54.366147 29930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.215.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.215.252@KRBTEST.COM" created.
I20260926 19:38:54.388435 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.369176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.6.215.254:40949)
0926 19:38:54.369427 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.369432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.370768 (+ 1336us) server_negotiation.cc:407] Connection header received
0926 19:38:54.372672 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.372678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.372815 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.372945 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.374475 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.375186 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.376822 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.377302 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.377447 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.381965 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.381974 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.381982 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.382033 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.384912 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.385904 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.385908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.385911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.386034 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.386530 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.386533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.386535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.386835 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.386941 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.387946 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.388137 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260926 19:38:54.389869 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.369791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40949 (local address 127.0.0.1:38652)
0926 19:38:54.370590 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.370631 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:54.372392 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:54.373184 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:54.373193 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:54.373743 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:54.374300 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.374317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.375378 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.375382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.376629 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.376645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.377593 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.377603 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.377968 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.378743 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:54.378782 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:54.381671 (+ 2889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:54.385064 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.385073 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.385091 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.385751 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.386159 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.386167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.386173 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.386413 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.386979 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:54.386990 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:54.387488 (+ 498us) client_negotiation.cc:769] Sending connection context
0926 19:38:54.387796 (+ 308us) client_negotiation.cc:240] Negotiation successful
0926 19:38:54.388350 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":324,"threads_started":1}
I20260926 19:38:54.391744 29930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:38:54.406594 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.392531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56440)
0926 19:38:54.392730 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.392747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:54.392907 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:54.393954 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:54.393958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:54.394393 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:54.394519 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.394528 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.395865 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.395873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.396702 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.396713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.397576 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.397581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.397765 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.398388 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:54.398405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:54.400652 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:54.403897 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.403903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.403907 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.404286 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.404655 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.404660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.404666 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.404877 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.405518 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:54.405525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:54.405676 (+ 151us) client_negotiation.cc:769] Sending connection context
0926 19:38:54.405826 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 19:38:54.406118 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 19:38:54.406626 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.392902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.6.215.253:46865)
0926 19:38:54.393553 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.393559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.393582 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:38:54.393624 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.393630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.393810 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.393966 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.394692 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.395725 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.396903 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.397434 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.397571 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.400903 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.400915 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.400929 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.400966 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.403756 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.404417 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.404434 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.404436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.404546 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.404975 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.404982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.404986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.405376 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.405598 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.405977 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.406235 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":506,"thread_start_us":366,"threads_started":1}
I20260926 19:38:54.408972 29930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:38:54.410072 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.409643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38670)
0926 19:38:54.409806 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.409933 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 19:38:54.410471 29881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.411919 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.411603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38680)
0926 19:38:54.411745 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.411824 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 19:38:54.412142 29930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.215.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab.
Entry for principal kudu/127.6.215.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab.
I20260926 19:38:54.420326 7007 external_mini_cluster.cc:1326] Running /tmp/dist-test-taske_ElXL/build/asan/bin/kudu
/tmp/dist-test-taske_ElXL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:40901
--webserver_interface=127.6.215.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab
--principal=kudu/127.6.215.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.215.212:34949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.215.254:40949,127.6.215.253:46865,127.6.215.252:40901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 19:38:54.461453 29930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 1
I20260926 19:38:54.462664 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.462211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38682)
0926 19:38:54.462408 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.462513 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 19:38:54.463754 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.463291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38688)
0926 19:38:54.463476 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.463572 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 19:38:54.463987 29930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.532240 29930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 2
I20260926 19:38:54.533514 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.532994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38698)
0926 19:38:54.533215 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.533329 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 19:38:54.534564 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.534189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38706)
0926 19:38:54.534353 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.534434 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 19:38:54.534778 29930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.561302 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } attempt: 5
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:38:54.576014 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.562347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56454 (local address 127.6.215.253:46865)
0926 19:38:54.562530 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.562535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.562554 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:38:54.562609 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.562613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.562688 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.562773 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.563756 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.564461 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.566322 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.566772 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.566901 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.570707 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.570712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.570716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.570746 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.573395 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.574267 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.574271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.574273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.574415 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.574809 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.574812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.574814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.575080 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.575187 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.575631 (+ 444us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.575795 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:38:54.576159 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.562082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56454)
0926 19:38:54.562317 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.562340 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:54.562478 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:54.562848 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:54.562854 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:54.563336 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:54.563566 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.563578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.564595 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.564602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.566132 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.566151 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.566937 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.566946 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.567282 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.567938 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:54.567960 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:54.570528 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:54.573543 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.573550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.573555 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.574147 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.574527 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.574533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.574540 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.574707 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.575193 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:54.575198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:54.575372 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 19:38:54.575533 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 19:38:54.575830 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 19:38:54.577240 29856 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } has no permanent_uuid. Determining permanent_uuid...
I20260926 19:38:54.578449 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.578078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38710)
0926 19:38:54.578239 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.578321 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 19:38:54.579384 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.579055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38718)
0926 19:38:54.579218 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.579288 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 19:38:54.579596 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.626050 29930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 3
I20260926 19:38:54.627183 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.626714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38726)
0926 19:38:54.626928 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.627019 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 19:38:54.628353 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.627922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38736)
0926 19:38:54.628152 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.628236 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260926 19:38:54.628603 29930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.630803 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 1
I20260926 19:38:54.631881 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.631445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38746)
0926 19:38:54.631652 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.631736 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 19:38:54.632882 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.632501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38754)
0926 19:38:54.632647 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.632721 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 19:38:54.633076 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.701335 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 2
I20260926 19:38:54.702487 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.701989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38762)
0926 19:38:54.702232 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.702323 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 19:38:54.703470 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.703121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38766)
0926 19:38:54.703270 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.703372 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 19:38:54.703663 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
W20260926 19:38:54.760632 29937 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 19:38:54.761042 29937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 19:38:54.761165 29937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 19:38:54.761314 29937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 19:38:54.768657 29937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 19:38:54.768806 29937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 19:38:54.768932 29937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 19:38:54.769018 29937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 19:38:54.769125 29937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 19:38:54.771831 29930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 4
I20260926 19:38:54.773046 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.772514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38770)
0926 19:38:54.772731 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.772891 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 19:38:54.774021 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.773701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38772)
0926 19:38:54.773861 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.773925 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 19:38:54.774245 29930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.779788 29937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.215.212:34949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.215.254:40949,127.6.215.253:46865,127.6.215.252:40901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/krb5kdc/kudu_127.6.215.252.keytab
--principal=kudu/127.6.215.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.215.252:40901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.215.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 18:32:33 UTC on ccycloud.quasar-tsfqhi.root.comops.site
build id 4675021
ASAN enabled
I20260926 19:38:54.781140 29937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 19:38:54.782542 29937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 19:38:54.794734 29943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:54.794677 29945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 19:38:54.795375 29942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 19:38:54.795329 29937 server_base.cc:1034] running on GCE node
I20260926 19:38:54.796286 29937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 19:38:54.797679 29937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 19:38:54.798918 29937 hybrid_clock.cc:648] HybridClock initialized: now 1790451534798847 us; error 38 us; skew 500 ppm
I20260926 19:38:54.799285 29937 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 19:38:54.802397 29937 init.cc:377] Logged in from keytab as kudu/127.6.215.252@KRBTEST.COM (short username kudu)
I20260926 19:38:54.803809 29937 webserver.cc:460] Webserver started at http://127.6.215.252:37549/ using document root <none> and password file <none>
I20260926 19:38:54.804520 29937 fs_manager.cc:359] Metadata directory not provided
I20260926 19:38:54.804643 29937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 19:38:54.805024 29937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 19:38:54.808001 29937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data/instance:
uuid: "c76d4ef8f8494a308fcc2f76fef6da01"
format_stamp: "Formatted at 2026-09-26 19:38:54 on dist-test-slave-phbh"
server_key: "3078744f1f80fde0c45931c2d91c824b"
server_key_iv: "2a49168d09fa7786861ecbec2cc3f4cf"
server_key_version: "encryptionkey@0"
I20260926 19:38:54.808846 29937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/wal/instance:
uuid: "c76d4ef8f8494a308fcc2f76fef6da01"
format_stamp: "Formatted at 2026-09-26 19:38:54 on dist-test-slave-phbh"
server_key: "3078744f1f80fde0c45931c2d91c824b"
server_key_iv: "2a49168d09fa7786861ecbec2cc3f4cf"
server_key_version: "encryptionkey@0"
I20260926 19:38:54.809857 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 3
I20260926 19:38:54.810848 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.810361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38780)
0926 19:38:54.810595 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.810672 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 19:38:54.811825 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.811435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38790)
0926 19:38:54.811646 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.811715 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260926 19:38:54.812013 29856 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.215.252:40901: Network error: Client connection negotiation failed: client connection to 127.6.215.252:40901: connect: Connection refused (error 111)
I20260926 19:38:54.814303 29937 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 19:38:54.817926 29952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 19:38:54.819365 29937 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 19:38:54.819557 29937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data,/tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/wal
uuid: "c76d4ef8f8494a308fcc2f76fef6da01"
format_stamp: "Formatted at 2026-09-26 19:38:54 on dist-test-slave-phbh"
server_key: "3078744f1f80fde0c45931c2d91c824b"
server_key_iv: "2a49168d09fa7786861ecbec2cc3f4cf"
server_key_version: "encryptionkey@0"
I20260926 19:38:54.819778 29937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 19:38:54.837237 29937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 19:38:54.841081 29937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 19:38:54.841365 29937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 19:38:54.860368 29937 rpc_server.cc:307] RPC server started. Bound to: 127.6.215.252:40901
I20260926 19:38:54.860387 30003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.215.252:40901 every 8 connection(s)
I20260926 19:38:54.861783 29937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/data/info.pb
I20260926 19:38:54.865814 7007 external_mini_cluster.cc:1388] Started /tmp/dist-test-taske_ElXL/build/asan/bin/kudu as pid 29937
I20260926 19:38:54.865994 7007 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taske_ElXL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790451176367082-7007-0/minicluster-data/master-2/wal/instance
I20260926 19:38:54.866207 30004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 19:38:54.866514 7007 external_mini_cluster.cc:1428] Setting key 1a525e6535aad7caee731be8f336a861
I20260926 19:38:54.874778 30004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
I20260926 19:38:54.889035 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.868167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.6.215.252:40901)
0926 19:38:54.869000 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.869012 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.869053 (+ 41us) server_negotiation.cc:407] Connection header received
0926 19:38:54.870944 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.870965 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.871362 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.871759 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.873263 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.874486 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.875746 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.876396 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.876594 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.881200 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.881207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.881217 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.881306 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.885086 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.885774 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.885789 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.885795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.885943 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.886305 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.886322 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.886330 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.886927 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.887240 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.887636 (+ 396us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.888023 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":361,"threads_started":1}
I20260926 19:38:54.891631 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.877086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.254:40949 (local address 127.0.0.1:38660)
0926 19:38:54.877798 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.877845 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:54.877986 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:54.878413 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:54.878418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:54.878965 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:54.879192 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.879200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.880367 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.880375 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.881714 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.881726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.882495 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.882502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.882718 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.883332 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:54.883372 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:54.885881 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:54.888962 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.888972 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.888979 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.889498 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.889884 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.889889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.889893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.890021 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.890567 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:54.890575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:54.890747 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 19:38:54.890939 (+ 192us) client_negotiation.cc:240] Negotiation successful
0926 19:38:54.891254 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":345,"threads_started":1}
I20260926 19:38:54.892362 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.877147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.6.215.254:40949)
0926 19:38:54.877384 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.877392 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.877949 (+ 557us) server_negotiation.cc:407] Connection header received
0926 19:38:54.878110 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.878116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.878201 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.878327 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.879350 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.880165 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.881892 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.882291 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.882389 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.886072 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.886079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.886085 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.886120 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.888830 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.889618 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.889624 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.889630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.889773 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.890158 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.890164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.890167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.890420 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.890584 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.891019 (+ 435us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.891562 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 19:38:54.893577 30004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.254@KRBTEST.COM
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:38:54.907733 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.894353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.253:46865 (local address 127.0.0.1:56462)
0926 19:38:54.894522 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.894540 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:54.894632 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:54.894971 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:54.894976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:54.895400 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:54.895571 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.895583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.896691 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.896695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.897886 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.897903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.898610 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.898616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.898811 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.899332 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:54.899360 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:54.902002 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:54.905204 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.905210 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.905217 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.905685 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.906068 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.906074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.906077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.906281 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.906799 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:54.906802 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:54.906940 (+ 138us) client_negotiation.cc:769] Sending connection context
0926 19:38:54.907094 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 19:38:54.907378 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 19:38:54.907763 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.894387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.6.215.253:46865)
0926 19:38:54.894580 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.894585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.894602 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:38:54.894769 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.894774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.894851 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.894950 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.895750 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.896548 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.898031 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.898450 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.898548 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.902180 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.902185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.902189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.902219 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.905051 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.905799 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.905805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.905809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.905947 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.906394 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.906398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.906400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.906691 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.906814 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.907189 (+ 375us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.907411 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 19:38:54.908465 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.891861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38672 (local address 127.6.215.254:40949)
0926 19:38:54.892544 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.892550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.892571 (+ 21us) server_negotiation.cc:407] Connection header received
0926 19:38:54.892628 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.892632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.892683 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.892825 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.894343 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.895205 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.896548 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.897108 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.897205 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.901952 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.901958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.901964 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.901999 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.904731 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.905559 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.905563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.905565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.905682 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.906864 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.906868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.906870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.907134 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.907415 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.907714 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.908116 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":310,"threads_started":1}
I20260926 19:38:54.908990 30004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.252@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.215.253@KRBTEST.COM
I20260926 19:38:54.923887 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.909912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38810)
0926 19:38:54.910198 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.910215 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:54.910326 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:54.910663 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:54.910667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:54.911113 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:54.911239 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.911247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.912485 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.912490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.913680 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.913692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.914618 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.914622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.914870 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.915445 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:54.915471 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:54.917731 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:54.921068 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.921076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.921083 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.921665 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.922009 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:54.922014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:54.922017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:54.922183 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:54.922763 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:54.922770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:54.922928 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 19:38:54.923112 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 19:38:54.923439 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260926 19:38:54.923889 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.909912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.6.215.252:40901)
0926 19:38:54.910094 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.910099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.910292 (+ 193us) server_negotiation.cc:407] Connection header received
0926 19:38:54.910448 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.910453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.910530 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.910623 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.911390 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.912359 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.913820 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.914433 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.914568 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.917922 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.917930 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.917936 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.917978 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.920938 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.921786 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.921789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.921792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.921899 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.922336 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.922340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.922343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.922641 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.922749 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.923249 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.923638 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":59}
I20260926 19:38:54.926558 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.910433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56472 (local address 127.6.215.253:46865)
0926 19:38:54.910572 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.910576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.910612 (+ 36us) server_negotiation.cc:407] Connection header received
0926 19:38:54.910654 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.910658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.910725 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.910837 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.912147 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.913200 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.914788 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.915427 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.915563 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.920573 (+ 5010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.920579 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.920582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.920614 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:54.923824 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.924577 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.924584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.924588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.924731 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:54.925143 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:54.925149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:54.925152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:54.925431 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:54.925709 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:54.926009 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 19:38:54.926253 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260926 19:38:54.930222 30004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: Bootstrap starting.
I20260926 19:38:54.931339 7007 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 19:38:54.933698 30004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:54.934725 30004 log.cc:824] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:54.933665 29831 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38672
I20260926 19:38:54.937513 30004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: No bootstrap required, opened a new log
I20260926 19:38:54.944380 30004 raft_consensus.cc:348] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:54.944739 30004 raft_consensus.cc:729] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c76d4ef8f8494a308fcc2f76fef6da01, State: Initialized, Role: FOLLOWER
I20260926 19:38:54.945235 30004 consensus_queue.cc:260] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:54.946117 30011 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:54.946455 30011 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:54.947419 30004 sys_catalog.cc:564] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 19:38:54.957186 30022 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 19:38:54.957316 30022 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 19:38:54.954834 29937 master_runner.cc:186] Error getting master registration for 127.6.215.254:40949: OK, instance_id { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" instance_seqno: 1790451533853695 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:38:54.966017 29937 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:38:54.989156 29856 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 4
Sep 26 19:38:54 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451533, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.254@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
I20260926 19:38:55.006062 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.989849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38822)
0926 19:38:54.990080 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:54.990106 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:54.990244 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:54.990667 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:54.990673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:54.991202 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:54.991418 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.991430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.992875 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.992883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.994392 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:54.994409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.995402 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.995409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:54.995598 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.996359 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:54.996383 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:54.999054 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:55.003155 (+ 4101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:55.003162 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:55.003166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:55.003681 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:55.004099 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:55.004104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:55.004109 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:55.004278 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:55.004959 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:55.004965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:55.005147 (+ 182us) client_negotiation.cc:769] Sending connection context
0926 19:38:55.005308 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 19:38:55.005627 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 19:38:55.006085 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:54.989934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.6.215.252:40901)
0926 19:38:54.990150 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:54.990154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:54.990173 (+ 19us) server_negotiation.cc:407] Connection header received
0926 19:38:54.990394 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:54.990404 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:54.990500 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:54.990613 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:54.991577 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.992693 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.994546 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:54.995243 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:54.995361 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:54.999266 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:54.999275 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:54.999280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:54.999331 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:55.002984 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.003818 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.003826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.003830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.003980 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.004388 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.004395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.004401 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.004824 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:55.004980 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:55.005423 (+ 443us) server_negotiation.cc:299] Negotiation successful
0926 19:38:55.005711 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 19:38:55.012001 29856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b: Bootstrap starting.
I20260926 19:38:55.016088 29856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:55.017431 29856 log.cc:824] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:55.021075 29856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b: No bootstrap required, opened a new log
I20260926 19:38:55.026593 29856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:55.026932 29856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd7ec5e8a8b4354ac0ae8545811a28b, State: Initialized, Role: FOLLOWER
I20260926 19:38:55.027386 29856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:55.028278 30025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.028728 30025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:55.029492 29856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:55.029587 29831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 19:38:55.029799 29831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:55.033257 29831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:55.034844 29831 leader_election.cc:288] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [CANDIDATE]: Term 1 election: Requested vote from peers ba5bae15240349a89603a589e0105a06 (127.6.215.253:46865), c76d4ef8f8494a308fcc2f76fef6da01 (127.6.215.252:40901)
I20260926 19:38:55.035998 29979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c76d4ef8f8494a308fcc2f76fef6da01"
I20260926 19:38:55.036559 29979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:55.040987 29979 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dd7ec5e8a8b4354ac0ae8545811a28b in term 1.
I20260926 19:38:55.041905 29807 leader_election.cc:302] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dd7ec5e8a8b4354ac0ae8545811a28b, c76d4ef8f8494a308fcc2f76fef6da01; no voters:
I20260926 19:38:55.042507 30025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 19:38:55.053673 30025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [term 1 LEADER]: Becoming Leader. State: Replica: 1dd7ec5e8a8b4354ac0ae8545811a28b, State: Running, Role: LEADER
I20260926 19:38:55.054586 30025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:55.063421 30032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd7ec5e8a8b4354ac0ae8545811a28b. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.064067 30032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:55.073477 29930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } attempt: 5
I20260926 19:38:55.074391 30040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 19:38:55 dist-test-slave-phbh krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790451534, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.215.253@KRBTEST.COM for kudu/127.6.215.252@KRBTEST.COM
I20260926 19:38:55.090574 30040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260926 19:38:55.092695 29810 tablet.cc:2369] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 19:38:55.093302 29958 tablet.cc:2369] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 19:38:55.100792 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:55.074183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.215.252:40901 (local address 127.0.0.1:38838)
0926 19:38:55.074595 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0926 19:38:55.074616 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 19:38:55.074752 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 19:38:55.075665 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 19:38:55.075670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 19:38:55.076171 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0926 19:38:55.076400 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:55.076416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.078497 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.078506 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:55.080480 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 19:38:55.080500 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.082060 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.082086 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 19:38:55.082487 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:55.083345 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 19:38:55.083398 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0926 19:38:55.088035 (+ 4637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 19:38:55.095302 (+ 7267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:55.095338 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:55.095342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:55.096151 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:55.096956 (+ 805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 19:38:55.096982 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 19:38:55.096986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 19:38:55.097243 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 19:38:55.098247 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 19:38:55.098253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 19:38:55.098509 (+ 256us) client_negotiation.cc:769] Sending connection context
0926 19:38:55.098725 (+ 216us) client_negotiation.cc:240] Negotiation successful
0926 19:38:55.099350 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280}
I20260926 19:38:55.101787 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:55.074979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38838 (local address 127.6.215.252:40901)
0926 19:38:55.075145 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.075150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.075167 (+ 17us) server_negotiation.cc:407] Connection header received
0926 19:38:55.075228 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.075233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.075305 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.075432 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.076674 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.078233 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.080740 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.081671 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.081796 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:55.088321 (+ 6525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:55.088329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:55.088334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:55.088385 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:55.095055 (+ 6670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.096450 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.096459 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.096462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.096740 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.097472 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.097477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.097481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.098061 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:55.099308 (+ 1247us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:55.099938 (+ 630us) server_negotiation.cc:299] Negotiation successful
0926 19:38:55.100734 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 19:38:55.107570 29930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06: Bootstrap starting.
I20260926 19:38:55.113677 29930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06: Neither blocks nor log segments found. Creating new log.
I20260926 19:38:55.115747 29930 log.cc:824] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06: Log is configured to *not* fsync() on all Append() calls
I20260926 19:38:55.118772 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:55.078321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.6.215.252:40901)
0926 19:38:55.095432 (+ 17111us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.095438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.096209 (+ 771us) server_negotiation.cc:407] Connection header received
0926 19:38:55.096271 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.096277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.096352 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.096455 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.098687 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.099977 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.105125 (+ 5148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.105903 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.106000 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:55.110855 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:55.110865 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:55.110870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:55.110912 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:55.114721 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.115664 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.115670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.115674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.115845 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.116500 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.116505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.116509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.116908 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:55.117148 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:55.118090 (+ 942us) server_negotiation.cc:299] Negotiation successful
0926 19:38:55.118323 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16962,"thread_start_us":17109,"threads_started":1}
I20260926 19:38:55.119297 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:55.072260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.6.215.254:40949)
0926 19:38:55.072517 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.072527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.081922 (+ 9395us) server_negotiation.cc:407] Connection header received
0926 19:38:55.082449 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.082456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.082544 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.082621 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.100908 (+ 18287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.101988 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.105604 (+ 3616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.106266 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.106371 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:55.111299 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:55.111306 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:55.111310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:55.111353 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:55.114989 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.116007 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.116012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.116018 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.116165 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.116703 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.116711 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.116716 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.117143 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:55.117405 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:55.118207 (+ 802us) server_negotiation.cc:299] Negotiation successful
0926 19:38:55.118483 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 19:38:55.123070 30050 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:38:55.125011 29930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06: No bootstrap required, opened a new log
I20260926 19:38:55.128158 29789 master_runner.cc:186] Error getting master registration for 127.6.215.253:46865: OK, instance_id { permanent_uuid: "ba5bae15240349a89603a589e0105a06" instance_seqno: 1790451534344418 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 19:38:55.142768 29789 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 19:38:55.142756 29930 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:55.143412 29930 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba5bae15240349a89603a589e0105a06, State: Initialized, Role: FOLLOWER
I20260926 19:38:55.144207 29930 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } }
I20260926 19:38:55.144840 29979 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [term 1 FOLLOWER]: Refusing update from remote peer 1dd7ec5e8a8b4354ac0ae8545811a28b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:55.145871 30032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:55.148962 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:55.100607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.6.215.253:46865)
0926 19:38:55.100762 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.100798 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.109458 (+ 8660us) server_negotiation.cc:407] Connection header received
0926 19:38:55.109755 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.109763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.109853 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.109935 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.123780 (+ 13845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.133056 (+ 9276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.135116 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:55.135858 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:55.135987 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:55.139768 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 19:38:55.139775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 19:38:55.139780 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 19:38:55.139820 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 19:38:55.143691 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.144554 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.144561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.144567 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.144725 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 19:38:55.145283 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 19:38:55.145291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 19:38:55.145296 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 19:38:55.145747 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 19:38:55.146022 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:55.147505 (+ 1483us) server_negotiation.cc:299] Negotiation successful
0926 19:38:55.148437 (+ 932us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 19:38:55.156737 29905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ba5bae15240349a89603a589e0105a06"
I20260926 19:38:55.157208 29905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [term 0 FOLLOWER]: Advancing to term 1
I20260926 19:38:55.160539 29905 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dd7ec5e8a8b4354ac0ae8545811a28b in term 1.
I20260926 19:38:55.161513 29904 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [term 1 FOLLOWER]: Refusing update from remote peer 1dd7ec5e8a8b4354ac0ae8545811a28b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 19:38:55.162921 30032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 19:38:55.169178 29930 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 19:38:55.173130 30011 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd7ec5e8a8b4354ac0ae8545811a28b. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.173532 30011 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:55.177923 30048 mvcc.cc:205] Tried to move back new op lower bound from 7333689487927005184 to 7333689487591309312. Current Snapshot: MvccSnapshot[applied={T|T < 7333689487927005184}]
I20260926 19:38:55.178308 30025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.178635 30025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:55.178996 30040 catalog_manager.cc:1347] Generated new cluster ID: 9283060a308e4187879acd513860d675
I20260926 19:38:55.179090 30040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 19:38:55.180521 30054 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.181020 30054 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:55.184000 30025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.184235 30025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b [sys.catalog]: This master's current role is: LEADER
I20260926 19:38:55.185199 30051 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd7ec5e8a8b4354ac0ae8545811a28b. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.185844 30055 mvcc.cc:205] Tried to move back new op lower bound from 7333689487927005184 to 7333689487591309312. Current Snapshot: MvccSnapshot[applied={T|T < 7333689487927005184}]
I20260926 19:38:55.185835 30051 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:55.187721 30054 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.187955 30054 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:55.188855 30052 mvcc.cc:205] Tried to move back new op lower bound from 7333689487927005184 to 7333689487591309312. Current Snapshot: MvccSnapshot[applied={T|T < 7333689487927005184}]
I20260926 19:38:55.204118 30011 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dd7ec5e8a8b4354ac0ae8545811a28b" member_type: VOTER last_known_addr { host: "127.6.215.254" port: 40949 } } peers { permanent_uuid: "ba5bae15240349a89603a589e0105a06" member_type: VOTER last_known_addr { host: "127.6.215.253" port: 46865 } } peers { permanent_uuid: "c76d4ef8f8494a308fcc2f76fef6da01" member_type: VOTER last_known_addr { host: "127.6.215.252" port: 40901 } } }
I20260926 19:38:55.204500 30011 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 19:38:55.205973 30070 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 19:38:55.214834 30040 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 19:38:55.216689 30040 catalog_manager.cc:1504] Loading token signing keys...
I20260926 19:38:55.243072 30040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dd7ec5e8a8b4354ac0ae8545811a28b: Generated new TSK 0
I20260926 19:38:55.244164 30040 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 19:38:55.323666 30045 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.314434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.6.215.252:40901)
0926 19:38:55.314643 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.314649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.321769 (+ 7120us) server_negotiation.cc:407] Connection header received
0926 19:38:55.322006 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.322014 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.322113 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.322183 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.323375 (+ 1192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38854: BlockingRecv error: recv got EOF from 127.0.0.1:38854 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260926 19:38:55.326256 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.322478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.6.215.253:46865)
0926 19:38:55.322661 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.322668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.324608 (+ 1940us) server_negotiation.cc:407] Connection header received
0926 19:38:55.324901 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.324906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.325000 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.325070 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.326028 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56488: BlockingRecv error: recv got EOF from 127.0.0.1:56488 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 19:38:55.328140 30009 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.314222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.6.215.254:40949)
0926 19:38:55.314416 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.314423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.321450 (+ 7027us) server_negotiation.cc:407] Connection header received
0926 19:38:55.321679 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.321686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.321792 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.321920 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.327879 (+ 5959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38704: BlockingRecv error: recv got EOF from 127.0.0.1:38704 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 19:38:55.338795 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.335099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.6.215.253:46865)
0926 19:38:55.335264 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.335271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.337250 (+ 1979us) server_negotiation.cc:407] Connection header received
0926 19:38:55.337484 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.337490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.337613 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.337728 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.338554 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56502: BlockingRecv error: recv got EOF from 127.0.0.1:56502 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 19:38:55.346719 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.344170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.6.215.253:46865)
0926 19:38:55.344325 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.344331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.345311 (+ 980us) server_negotiation.cc:407] Connection header received
0926 19:38:55.345524 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.345530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.345620 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.345728 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.346507 (+ 779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56508: BlockingRecv error: recv got EOF from 127.0.0.1:56508 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260926 19:38:55.383116 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.380967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.6.215.253:46865)
0926 19:38:55.381154 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.381161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.381560 (+ 399us) server_negotiation.cc:407] Connection header received
0926 19:38:55.381776 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.381782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.381885 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.381995 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.382878 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56510: BlockingRecv error: recv got EOF from 127.0.0.1:56510 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260926 19:38:55.393875 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.391630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.6.215.253:46865)
0926 19:38:55.391807 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.391827 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.392338 (+ 511us) server_negotiation.cc:407] Connection header received
0926 19:38:55.392535 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.392542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.392645 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.392786 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.393647 (+ 861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56524: BlockingRecv error: recv got EOF from 127.0.0.1:56524 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 19:38:55.414291 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.405539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56528 (local address 127.6.215.253:46865)
0926 19:38:55.405722 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.405729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.406270 (+ 541us) server_negotiation.cc:407] Connection header received
0926 19:38:55.406448 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.406454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.406549 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.406657 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.414055 (+ 7398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56528: BlockingRecv error: recv got EOF from 127.0.0.1:56528 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 19:38:55.434643 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.432650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.6.215.253:46865)
0926 19:38:55.432913 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.432918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.432943 (+ 25us) server_negotiation.cc:407] Connection header received
0926 19:38:55.433130 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.433137 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.433246 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.433317 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.434384 (+ 1067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56530: BlockingRecv error: recv got EOF from 127.0.0.1:56530 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 19:38:55.473249 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.471104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.6.215.253:46865)
0926 19:38:55.471298 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.471305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.471569 (+ 264us) server_negotiation.cc:407] Connection header received
0926 19:38:55.471803 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.471809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.471910 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.472029 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.472991 (+ 962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56534: BlockingRecv error: recv got EOF from 127.0.0.1:56534 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260926 19:38:55.543329 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.541309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.6.215.253:46865)
0926 19:38:55.541531 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.541536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.541861 (+ 325us) server_negotiation.cc:407] Connection header received
0926 19:38:55.542063 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.542067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.542138 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.542223 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.543108 (+ 885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56548: BlockingRecv error: recv got EOF from 127.0.0.1:56548 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260926 19:38:55.677019 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.675092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56564 (local address 127.6.215.253:46865)
0926 19:38:55.675267 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.675272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.675623 (+ 351us) server_negotiation.cc:407] Connection header received
0926 19:38:55.675783 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.675787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.675859 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.675954 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.676757 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56564: BlockingRecv error: recv got EOF from 127.0.0.1:56564 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 19:38:55.938728 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 19:38:55.936821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.6.215.253:46865)
0926 19:38:55.937083 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:55.937090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:55.937347 (+ 257us) server_negotiation.cc:407] Connection header received
0926 19:38:55.937583 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:55.937589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:55.937674 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:55.937800 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 19:38:55.938561 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56578: BlockingRecv error: recv got EOF from 127.0.0.1:56578 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260926 19:38:55.959403 30022 catalog_manager.cc:1259] Loaded cluster ID: 9283060a308e4187879acd513860d675
I20260926 19:38:55.959568 30022 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: loading cluster ID for follower catalog manager: success
I20260926 19:38:55.962337 30022 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: acquiring CA information for follower catalog manager: success
I20260926 19:38:55.964205 30022 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c76d4ef8f8494a308fcc2f76fef6da01: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:38:56.210186 30070 catalog_manager.cc:1259] Loaded cluster ID: 9283060a308e4187879acd513860d675
I20260926 19:38:56.210301 30070 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06: loading cluster ID for follower catalog manager: success
I20260926 19:38:56.212924 30070 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06: acquiring CA information for follower catalog manager: success
I20260926 19:38:56.214756 30070 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ba5bae15240349a89603a589e0105a06: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 19:38:56.472533 30137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:56.465278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.6.215.253:46865)
0926 19:38:56.466181 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:56.466187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:56.466210 (+ 23us) server_negotiation.cc:407] Connection header received
0926 19:38:56.466295 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:56.466301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:56.466500 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:56.466649 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:56.467058 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.468410 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.469745 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.470427 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.470554 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:56.471068 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:56.471858 (+ 790us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:56.472035 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:56.472087 (+ 52us) server_negotiation.cc:299] Negotiation successful
0926 19:38:56.472176 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":628,"threads_started":1}
I20260926 19:38:56.488541 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:56.479773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.6.215.252:40901)
0926 19:38:56.482611 (+ 2838us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:56.482616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:56.482631 (+ 15us) server_negotiation.cc:407] Connection header received
0926 19:38:56.482737 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:56.482740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:56.482904 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:56.483043 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:56.483469 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.484155 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.486101 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.486510 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.486606 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:56.487221 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:56.487890 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:56.488002 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:56.488146 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 19:38:56.488211 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2742,"thread_start_us":2616,"threads_started":1}
I20260926 19:38:56.514489 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:56.495253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.6.215.254:40949)
0926 19:38:56.505740 (+ 10487us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:56.505746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:56.505772 (+ 26us) server_negotiation.cc:407] Connection header received
0926 19:38:56.506109 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:56.506116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:56.506397 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:56.507245 (+ 848us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:56.507427 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.508524 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.510733 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.511401 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.511848 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:56.512878 (+ 1030us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:56.513417 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:56.514015 (+ 598us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:56.514081 (+ 66us) server_negotiation.cc:299] Negotiation successful
0926 19:38:56.514176 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10332,"thread_start_us":10697,"threads_started":1}
I20260926 19:38:56.515925 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:56.495832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.6.215.252:40901)
0926 19:38:56.496002 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:56.496008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:56.506698 (+ 10690us) server_negotiation.cc:407] Connection header received
0926 19:38:56.507025 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:56.507031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:56.507134 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:56.507220 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:56.508160 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.509568 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.512465 (+ 2897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.513266 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.513420 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:56.515047 (+ 1627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:56.515187 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:56.515317 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:56.515550 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 19:38:56.515636 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 19:38:56.520413 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29789
I20260926 19:38:56.520905 30137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 19:38:56.506839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.6.215.253:46865)
0926 19:38:56.507029 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 19:38:56.507034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 19:38:56.514979 (+ 7945us) server_negotiation.cc:407] Connection header received
0926 19:38:56.515291 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 19:38:56.515297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 19:38:56.515389 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 19:38:56.515497 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 19:38:56.515884 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.517019 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.518834 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 19:38:56.519409 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 19:38:56.519543 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 19:38:56.519989 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 19:38:56.520138 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 19:38:56.520260 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 19:38:56.520411 (+ 151us) server_negotiation.cc:299] Negotiation successful
0926 19:38:56.520494 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 19:38:56.521646 29851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:56.638330 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29789
I20260926 19:38:56.659966 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29863
I20260926 19:38:56.661067 29925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:56.767048 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29863
I20260926 19:38:56.786908 7007 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taske_ElXL/build/asan/bin/kudu pid 29937
I20260926 19:38:56.788029 29999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 19:38:56.894637 7007 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taske_ElXL/build/asan/bin/kudu with pid 29937
2026-09-26T19:38:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4880 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9280 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (360495 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST